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Abstract 


An investigation is made into the problem of constructing 
a model of the appearance to an optical input device of scenes 
consisting of plane-faced geometric solids. The goal jis to 
study algorithms which find the real straight edges in the 
scenes, taking into account smooth variations in intensity over 
faces of the solids, blurring of edges and noise. A general 
mathematical analysis is made of optimal methods for identify- 
ing the edge lines in figures, given a raster of intensities 
covering the entire field of view. There is given in addition 
a suboptimal statistical decision procedure, based on the model, 
for the identification of a line within a narrow band on the 
field of view given an array of intensities from within the 
band. A computer program has been written and extensively 
tested which implements this procedure and extracts lines 
from real scenes. Other programs were written which judge 
the completeness of extracted sets of lines, and propose and 
test for additional lines which had escaped initial detection. 
The performance of these programs is discussed in relation to 
the theory derived from the model, and with regard to their 
use of global information in detecting and proposing lines. 


*This report reproduces a thesis of the same title submitted 
to the Department of Mathematics, Massachusetts Institute of 
Technology, in partial fulfillment of the requirements for 
the degree of Doctor of Philosophy. 
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CHAPTER 1 
TOWARD A THEORY OF THE OPTIMAL USE OF INTENSITY 
INFORMATION IN THE DETECTION OF LINES IN A 


VISUAL FIELD 


1s INTRODUCTION AND DEFINITIONS 


We are interested in studying procedures which identify 
the images of straight lines in some two-dimensional (picture-like) 
projection of an array of real objects, given some large set of 
intensity values from points on the two-dimensional projection 
plane. We shall confine our attention to arrays of plane-faced 
solids. Applying a line-detection procedure to the image of an 
array of such objects yields a ''description"” of the objects which 
is, in effect, a line drawing of the objects from a particular 


point of view. Such a description is complete in the sense that it 
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contains all information obtainable from that point of view which is 
relevant to determining the shape, location in space, and so forth 
of the objects. Information of the latter sort, which may be 
regarded as constituting a more detailed description of the array 
of objects, may be derived from one or several "line drawings" 

of a particular array of objects, by methods surveyed in the last 
chapter of this paper. 

In the first chapters of this paper, we shall investigate the 
problem of constructing line detection procedures which make 
optimal use of the set of intensity values to which they are 
applied, by being maximally accurate, in various senses, about 
the lines they claim to exist. Such investigations require a 
formalism, definitions and so forth, which will presently be 
developed in this section. In the meantime we shall informally 
discuss several considerations relevant to the development of 
such a theory, with the idea of motivating some of the formal 
presentation of the rest of the chapter. 

First, it appears that a theory of optimal edge finding 
procedures would be intractably difficult if we were to consider 
procedures which decide on the locations at which to measure 


each successive intensity value. This leads into the realm of 


sequential decision procedures; and we should not attempt this 
until we first have a reasonable understanding of the simpler 
non-sequential case. One may get an idea of the inherent 
complexity of optimal sequential decision procedures by 
consulting, e.g., Bellman (Bellman). For the moment, then, 


we shall assume that the intensities are obtained from the two- 


dimensional projection of the array of objects in a predetermined 


pattern, such as at the vertices of a grid of squares, and that the 


line finding procedure acts on exactly these intensities. We 
shall return to the problem of sequential analysis of scenes in 
chapter four, where the matter is approached at the heuristic 
level. 

A second assumption to be made concerning the set of 
intensities upon which a line finding procedure acts concerns 
the relation between the actual intensity at a point in the two 
dimensional view of the scene and the intensity available to 
the line-finding procedure, which would be obtained by a real 
optical input device. In general such a device is subject to 
various forms of optical blurring and distortion, as well as to 
random noise. If there is nothing known about the noise, then 
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the set of intensity values obtained by such a device contains no 
information about the array of objects at hand, in particular 
about the locations of the projections of their edge lines. It is 

a reasonable assumption that the obtained values of intensity at 
any point be normally distributed about the actual intensity, and 
that the standard deviation of such a distribution be a function 

of the intensity at the point. It is further reasonable to assume 
that any spatial distortion of the intensities be accountable for 
principally as some sort of blurring phenomenon. These forms 
of distortion and noise are present in any real optical or electro- 
optical system; and certain devices, such as an image-dissector, 
may be designed so that these are the principal effects. 

A third restriction involves the relation between the size 
of the objects in the scenes under consideration, or more 
properly the size of the images of these objects in a two 
dimensional projection, and the spatial density of points at which 
intensities are measured. If the intensities are obtained in a 
Simple grid pattern with a spacing of one unit on the image plane, 
it would clearly be impossible to detect edge lines whose images, 
for example, were shorter than one unit. In judging the 
optimality of an edge detector, it would surely be unfair to 
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penalize it for not detecting so short aline. A solution to the 
problem might be to consider only arrays of objects whose 
images are larger than a certain size relative to the spacing of 
the points at which intensities are measured. An alternative 
approach, which will actually be employed, is as follows: Choose 
a Suitable geometry of points in the image space at which to 
measure intensities. Then define a simple predicate on exact 
values of intensity which would be obtained at these points in the 
absence of all noise, distortion and imperfections in the objects 
being examined. Then use this criterion for the existence of a 
line to judge the performance of a line detection procedure which 
measures intensities at the same points, but with a real optical 
input device. By this means, one is essentially determining how 
well the procedure "sees through" the distortion and noise; and does 
not penalize it for not "seeing" lines that it could not possibly 
detect from intensities measured at points of the given grid 
even in the absence of noise. 

The concepts so far discussed may be summarized in the 
following definitions. 
Definition: an intensity function, I(x, y), is a real-valued 

function whose values and derivatives are defined 


12 


Definition: 


Definition: 


Definition: 


Definition: 


Definition: 


Definition: 


on a square in 2-space, except for a set of 
measure 0. 

a description space, D, is an arbitrary set of 
elements {dj}. 

a sampling set, 5, is a finite set of points 

{(xpyp). --. } 

a noise-free sample, I(S), of an intensity function 
I(x, y), is the set of values of I(x;, yj) for the 

points (x;, y;) of S. 

a noisy sample, J(5), is defined as above, but with 
the values of intensity modified by some distortion 
and noise functions. 

an identification function, F(I), is a mapping from 
a set of intensity functions onto some description 
space. 

F: {If >D 

a noise-free sample identification function, G(I(S)), 
is a mapping from a set of noise-free samples onto 
a description space. 


G: {1(S)} >D 
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Definition: a noisy-sample identification function, H(J(S)), 
is a mapping from a set of noisy samples to a 
description space. 
H: {J(S)} >D 
Definition: asample identification function G, together with a 
space of noise-free samples {1(s)} and a description 
space D are said to be descriptively complete with 
respect to an identification function F if 
F(I)=d; <> G(I(S))=d; 
This definition describes the situation where all that one wishes to 
say about a scene may be inferred from the values of intensity at 
a predetermined finite set of locations on the visual field. 
Definition: an intensity function I*(x, y) on a domain A is said 
to be equivalent under the blurring function 
B(x, y) to a function I(x, y) if: 
T*(x, y)= fr v)B(u-x, v-y)du dv 
A 
Definition: an intensity function I**(x, y) is said to be 
Gaussian-noise modulated with respect to an 
intensity function I*(x, y) if successive values of 
I¥*(x0, Yq) distribute normally about T¥(xp, Yo) with 
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a standard deviation which is dependent solely on 
the value of I*(x, Yo): 

Definition: a transducer which goes from an intensity signal 
I(x, y) to a Gaussian-noise modulated function 
[**(x, y) of a function I*(x, y) which is equivalent 
under blurring to I(x, y), is called a transducer 
subject to fundamental distortion. 

Definition: ascene system with identifiers consists of: 

{1 (x, y)}, a set of intensity functions on a common 
domain; 

{(x,, y)} = §, a sampling set; 

{1,(S)}, a set of noise-free samples, one for each 
intensity function 1;; 

{J,(S)f, a set of noisy samples, an infinite number 
of J,(S) for each 1,(S). The relation between a set 
of J;(S) and the corresponding I,(S) is entirely - 
accountable for by fundamental distortion. 

{d;} = D, a description space; 

F;: {I(x, y)} —D, an identification function; 

G: {1,(S)} —+>D, a noise-free sample identification 
function; 

H: { 5,(S)} —>D, a noisy-sample identification function. 
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Finally, for completeness: 

Definition: ascene system with identifiers is said to be 
descriptively complete if the relevant parts conform 
to the descriptive completeness criterion above. 

The various restrictions set forth in the earlier parts of 
this section may be simply summarized by saying that we will 
concern ourselves exclusively with scene systems with identifiers 


which are descriptively complete. 


IT. COMPLETE SYSTEMS OF STRAIGHT -LINE SCENES 


A criterion for what constitutes a line in a noise-free 
Sample will now be given. This definition will provide a basis 
for a definition of a line in an intensity function which will be 
appropriate for ensuring completeness. 

Prior to making this definition, however, it is necessary 
to digress slightly to consider a plausible restriction on the 
geometry of the sampling set. An important desideratum of any 
line finder is that it not be especially sensitive to lines ina 
particular part of the field of view, or at any particular 
orientation in the field of view. It may be that of two procedures, 


one will find more lines with a given amount of effort, but at the 
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expense of uniformity in this sense. One would still not prefer 
the ''better'"' line finder, on account of its inhomogeneous or 
anisotropis behavior. One thus wishes to guarantee that any line 
finding procedure that is a candidate for being considered optimal 
is in addition uniform in this sense. A necessary condition seems 
to be to have a sampling set which is distributed uniformlu over 
the entire field of view. Without proving this last assertion, we 
shall henceforth restrict our attention to systems whose sampling 
sets consist of points on all vertices of some grid of squares 
covering the entire field of view. 

A straight line in a noise-free sample over a sampling 
region as described above may be defined in terms of the notions 
of discontinuity and anomaly. We define three such notions 
for, e.g., a set of colinear intensities ij, ig, ig and ig, using 
constants K, K' and K': 

1) A discontinuity of intensity occurs at a point between the 
points at which ig and ig were obtained if: 

lig - ig} > K(liq - ig] + lig - ig). 

2) An anomalous intensity occurs at the ‘point where i, 
was obtained if: 

ety. i W2heK' 
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3) A discontinuity in shading occurs at a point between 
the points at which iy and i, were obtained if: 

Iti, - i.) ~ Gg - 4,)| > K". 

A line may be defined as two or more such points, all of the 
same type and all adjacent and colinear. 

It is now easy to specify a class of systems with identifiers 
which are descriptively complete in their identification of lines in 
ascene. One may simply choose a scene system with a description 
Space appropriate for describing lines, and define a noise-free 
sample identification function G in such a way that it appropriately 
identifies all the lines in a noise-free sample according to the above 
definition of aline. One may then define the identification function 
F as follows: 

F(I)=d; <=> G(I(S))=dj. 

The completeness follows a fortiori. 

In whatever system we shall henceforth be considering, we 
shall assume that the description space is the power set of the set 
of all lines definable as above on noise-free samples. We shall 
further assume that the noise-free sample identification function 
of the system is as defined in the previous paragraph. The 
choice of an optimal noisy-sample identification function for this 


type of system is the subject of the remainder of this paper. 
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Ill. OPTIMIZATION CRITERIA 


The variables to be considered in discussing optimal 
object recognition algorithms are number of intensities read, 
time taken to read each intensity value, length of computation 
time, and rate of error in identifying lines. We will not be 
concerned with computation time, as this would require more 
knowledge about ''mimimal" versions of the algorithms concerned 
than is available today. On the other hand, the number of points 
at which intensities are taken, and the length of time taken in 
getting the intensities can be related in a natural way to the 
accuracy of identifying and locating lines, or other features 
ofa scene. Intuitively, there must exist a trade-off between 
the measurement cost of a certain amount of intensity 
information; and accuracy, measured in terms of number 
of lines correctly identified. Optimization may consist in 
minimizing some type of error rate, with a fixed overhead 
in effort. One may, alternatively, wish to set a certain 
error rate criterion, and minimize the amount of effort 


necessary to achieve it. It is easy to see that this latter 
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form of optimization may be reduced to the former: Suppose there 
exists a procedure P(E) which, using E units of effort, minimizes 
the error rate. If it is desired to minimize effort for a fixed 
error rate, one may simply consider all possible instances of 

the procedure P(E) with error rate below the desired one, and 
choose the procedure P(E*) where E* is a minimum among all 
such amounts of effort. This procedure uses the least possible 
amount of effort while guaranteeing the error rate to be below a 
certain value. We may thus without loss of generality restrict our 
attention to procedures which make a minimum of errors for a 
given amount of overhead. 

Further discussion of error rate requires a more detailed 
discussion of various types of classification errors and the 
probabilities with which they occur. We shall base this discussion 
on the notion of false positive error, the assertion by a line finder 
that a line exists at a particular place on the visual field when no 
such line exists in the scene under consideration, and upon the 
converse notion of true negative error. We will begin with some 
definitions regarding line identification functions. 

Definition: In a system of the type under consideration, for 
some noise-free sample I,(S) and a corresponding 
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noisy pattern Jj(S), we call the set of lines 
H(J(S))-G(E(S)) the set of false positive errors 
in the identification of the scene from the noisy 
sample J;(S). 

Definition: For similar conditions, the set G(T;(S))-H( (8) 
is called the set of true negative errors in the 
identification of the scene from the noisy sample 
Jj(S). 

Definition: Let A and B be two sets, then the difference 
cardinality, N(A,B), is the cardinality of the 
set A-B. 

It follows immediately that for any J,(S) and corresponding 

I,(S), the number of false positive errors is given by: 

N(H(S (8), GAT(8))), 
and the number of true negative errors by: 

N(G(I,(S)), H(J.(S))). 

With the aid of the function N defined above, we may 
indicate the performance of a noisy sample identification function 
in terms of the number of errors of each type it will make in 
describing any particular noisy sample. Before being able to 
indicate the over-all performance of a noisy sample identification 
function, one must as well be able to specify the relative 
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probabilities of occurrence of the various pairs of noise-free and 

noisy samples. For the present we need only the first of the 

following definitions; however they will all ultimately be necessary. 

Definition: P(I(S), Ji(8)) is the joint probability of occurrence 
of the noise-free sample I,(S) and the noisy sample 
J,(8). 

Definition: P(J,(S)/1;(S)) is the conditional probability that the 
noisy sample J,(S) will occur given the occurrence 
of noise-free sample I,(S). 

Definition: P(I;(S)/J5(S)) is defined similarly. 

Definition: P(I,(S)), P(J;(S)) are the a priori probabilities of 
occurrence of the noise-free sample I(S) and the 
noisy sample J,(S). 

We may now discuss the expected rate of error of a 

classification system of the type under consideration. 

Lemma: The false-positive error rate, or expected number 
of false positive errors made by a complete scene 
system is given by: 


> PU,(S), J (SINGH, (S)), G;(S)) 
inj 


Proof: Whenever the noisy sample J,(8) occurs in 
conjunction with the scene representable by 
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the noise-free sample IS), the number of 
false positive crrors is: 


NCT(S,(S)), G(I.(S))). 


1 


This occurs with probability 


PAT(S), J en): QED 
Lemma: The truce negative error rate in a similar situation 


is piven by: 


>, PU CS), J (S)IN(GG,(S)), HG 
in 


the proof is similar. 
With the aid of the following definitions, we may define two 
additional error-rate measures: 


Definition: M(A,B), the total number of errors is defined by: 


MCA, B)=N(A, B)+N(B, A). 


Definition: O(A, B), the identity predicate, is defined by: 


O(A, B)- 1 if A-b, 
O(A, B)=0 if AFB. 

Lemma: The correct recognition rate, or expected rate at 
Which a scene is perfectly identified is given by 
the following: 


>, PUL(s), J (SNO(GG (S)), H(L(S))), 
i] , 


the proof is as above. 
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Theorem: The total error rate, or expected number of errors 
made by a complete scene system is given by: 


D PULS), J,(SM(G(I;(8)), HL(S))), 
i, J 


the proof is as above. 
As acorrolary to the above theorems, we make the 

following definitions. 
Definition: — a(J;(S), d,)= 2.PU(S5(S)NG, GAI,(S))) 

is the expected rate of false positive errors on 

occasions when the noisy sample Jj(S) occurs, 

provided the description d, is assigned to it. 

finition: b(J. _)= : : : ; 

Definition: — b(J,(S), d,) 2, PU(S)/ TSN (SD) di) 

is the expected error rate of true negative errors 


under similar conditions. 
IV. OPTIMIZATION PROCEDURES 


With the aid of the definitions in the previous section, we 
are in a position to discuss four optimization procedures. The 
first involves the maximization of the probability of correctly 
recognizing a scene in its entirety. The second involves 
minimizing the sum of the false positive and true negative error 
rates. The last two involve the maximization of one of the error 
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rates with the other error rate held constant. Since these latter 

two procedures are mathematically nearly equivalent, we shall 

consider only the case of holding the false positive error rate 
constant, and minimizing the true negative error rate. 

Theorem: The scene recognition procedure which maximizes 
the rate of correct recognition of scenes in their 
entirety is given by the function H(J;)=dy. such that 
DOGU,(S)), 4, YPU,(S), Jj(S)) 
ie maximized. 

Proof: If description dy. is assigned to the noisy sample 
J,(8), then a perfect recognition of the scene will 
occur whenever L(S), the corresponding state of the 
real world, has the property that G(I,(S))=d,. This 
occurs with a probability P(I(S)/J5(S)) for a particular 
I,(S) having this property. Hence the expected rate 
of correct identification for the noisy sample 
a) is: 


>, P(Iy(S)/54(S)). 
GU, (S))= 4, 


This is equivalent to: 

DAG CS), dy IPT ;(S)/I(8)) 
i 

or: 


1/P(T(S)) DL O(GU{(S)), d PUES), J (8). 
L 
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Theorem: 


Proof: 


For a given Ji(S), the rate of correct recognition 
is obtained by maximizing this sum. This 
optimization for a particular J,(S) is independent in 
its effect on optimizing the recognition of other 
J,(S). Hence this noisy-sample-wise optimization 
procedure provides an optimum recognition procedure 
for the set of all a tee 

The scene recognition procedure which minimizes 
the rate of false positive errors plus true negative 
errors is given by the function H(J(S))= dy. with the 
property that: 

2 MGS), d, P(I,(S), J,(S)) 

is minimized. 


Similar to that of the preceding theorem. 


The last case is a bit more complex. 


Theorem: 


Suppose it is desired to keep the false positive error 
rate at or below a constant value a, while minimizing 
the true negative error rate. Then there exists a 

c, depending only on a, with the property that the 
desired optimality may be achieved by assigning to 


J5(S) the description d.. such that 
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Proof: 


c-a(Ji(S), d.) + (1-c) b(S;(S), dy) 

is minimal for all dj. 

1) There must exist a description function having 
false positive error rate at or below a, and minimal 
true negative error rate. This follows from the 
fact that, e.g., H(J;(S)) = 0 gives a false positive 
error rate of 0. Hence the set of decision functions 
with false positive error rates belowa@is non-empty. 
There must exist one with minimal true negative 
error rate. Let it be denoted by H(Ji(S)), and its 
true negative error rate by 8B. 

2) Consider a decision function Hx(J;(S)) which 
assigns to Ji(8) the description d, such that 
x-a(J,(S)), di.) + (1-x) b(5(S), d,) is mibimal for all 


dj. 


We shall show that for a suitable value c of x, 
the function H&(S;(S)) has a false positive error rate 
at or below a, and a true negative error rate of B. 
From this it will follow that the function Hx (J;(S)), 
which is the function referred to in the theorem, has 


the desired optimality. 
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3) Let: 

a(j)= a(Jj(S), H(J, (S))), 

b(j)=b(J(S), H(J;(S))); and also 

ax(j)=a(J,(S), H(J i(S))), 

bx(J)=b(JT (8), Hx(J,(S))). 

From the definition of Hx(Jj(S)), we know that for 
all j, 

x-a(j) + (1-x) b(j) > x-a(j) + (1-x) b&(j). 
Multiplying by P(J;(S)) and summing on j, we have: 


see) 4 Be 


x > ax(GP(T,(S) + (1 -x)) /bx(J)P(T,(S)). 
j j 


But, by definition, 


DAjP(T(S)) = @, and 


J 


> p(P(TAS)) = B. 
j J 


Also, letting 


ZY = a, and 
DPRIPC;(S) = Be, 

J 

we have: 


xat+ (1-x)B 2 xark + Bx. 
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4) It is not difficult to see that x may be chosen so 
as to make as =a, This follows by assuming that 
ax is a continuous function of x; and noting that 

ie | = 0, since Hy minimizes false positive errors 
ignoring true negative errors, and H(J;(S)) = @ 
achieves a false positive error rate of 0. By similar 
reasoning, a¥ increases without bound as x decreases 
from 1. Thus for some value c of x the value of 


a* =a, This approach may be applied to the 


: 
discrete case, where at least ae will be very close 
toa. The expression of the previous step will then 
reduce to: 

ca+ (1l-c)B >ca+t (1-c)px. 


But B is minimal, and (1-c)>0, thus: 


B= p*. QED 


29 


CHAPTER 2 
TH OPTIMAL USE OF REGIONAL INTENSITY INFORMATION 


IN THE DETECTION OF LINES 


I. THE REGIONAL GUALITY OF ISOLATED LINES 


In this section we shall direct our attention to Situations 
where there exist no systematic spatial relationships among the 
lines appearing in scenes under consideration. We shall bricfly 
discuss the extent to which lines in this case are "regional" in the 
sense that all intensity information relevant to the detection of a 
line is contained on the visual field within a certain short distance 
from the line. 

Definition: Let 5 be a sampling region consisting of points 
uniformly distributed over a field of view. Let 
ICA, B, 5) be a predicate on intensity values at points 
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of S, which attempts to decide whether or not there 
is a line segment through points A and B. Let a@ 
and B respectively be the false positive and true 
negative error rates of the function F(A, B, S). 
Then the quantity 55.(€) is defined as the smallest 
distance such that there exists a predicate F*(A, B) 
which decides on the existence of a line through A 
and B only on the basis of intensities of S lying 
within this distance of AB, and which has false 
positive and true negative error rates respectively 
below at+eée, and Bre. 
For some small value of €, the value of 6 (€) may take ona 
wide variety of values in different sorts of situations. For example 
5 al €) would be large in the situation where scenes consist of 
rectangular parallelapipeds. In this case, lines parallel to a 
proposed line, or incident with an end, constitute useful evidence 
for the existence of the proposed line. The spatial remoteness of 
useful evidence in this case might make 5(€) not appreciably 
smaller than the radius of the field of view for small values of e€. 
However, 5-(€) may be relatively small in the case of scenes 
consisting of single discontinuities of intensity across the field, 
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or of sets of straight lines drawn at random across a sheet of paper. 
A hypothetical situation of particular interest is that in which scenes 
consist of lines which, within a relatively large radius, behave like 
lines given rise to by edges of plane-faced solids; but which are 
randomly distributed across the field of view. In cases such as 
this, two effects limit the predictive value of intensity information 
remote from a proposed line. 

The first effect has to do with the unpredictibility of 
intensity values away from an edge. In the absence of optical 
distortion and noise, intensities on a line normal to an edge might 


appear as in Figure 2.1. The determination of the existence of an 


HYPOTHETICAL IDEALIZED 
INTENSITY PROFILE AT 
AN EDGE (Xq) 


Figure 2.1 


intensity discontinuity at x, is dependent upon.finding the right and 


0 


left limits of the intensity function at the point x The intensities 


0° 
32 


along the normal away from Xo contain information about the left 
and right limits at x5. However the amount of information contained 
in these intensities falls off as one goes farther from the edge. This 
is due to possible unpredictible non-uniformities in the illumination 
and reflectivity of the objects in the scene. 

The second effect has to do with the nature of the blurring 


inherent in an optical system. An edge subject to such blurring 


appears as in Figure 2.2. An abrupt step function is smeared out 


INTENSITY PROFILE AT AN 
EDGE (X9) SUBJECT TO 
BLURRING 


Xo x 

Figure 2.2 
a small distance from the edge and causes extreme values of the 
various derivatives of the intensity function in a small neighborhood 
of the edge. It is the extreme values of the derivatives that 
characterize and identify an edge. However, the effect of the 
existence of an edge on the derivatives falls off drastically as one 
goes away from the edge. Hence, derivative information on a set 


of intensities remote from a possible edge can give little 
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information about the presence or absence of the edge. 

The spatial extent of this effect is determined by the 
critical resolution radius defined as follows. 

Definition: The critical resolution radius is the radius of a 
circle whose intensity equals (I+J)/2 around the 
image of a point of intensity I on a background of 
intensity J. 

This radius is related to such factors as the resolving power of 

the optics of the system. Empirical evidence suggests that for 

relatively small values of €, the value of 5,,(€) is on the order of 

magnitude of a few times this radius, in the hypothetical case 
mentioned of randomly distributed parallelopiped-edge-like lines. 
In situations where the existence of a line is indeed 
virtually determinable entirely from intensity information contained 
within a few critical radii of the proposed line, we can test for the 
presence of a line in a given location by examining an area which 
could only possibly contain a few other lines resolvable from the 
given line. In general there would seldom be any line but the 
central line in an examined region. This suggests the development 
of a theory of the optimal use of intensity information within a 


narrow rectangular region, based on a model which assumes the 
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possible existence of at most one line in the region. 
II. A REGIONAL VERSION OF THE GENERAL THEORY 


A regional version of the theory presented in the previous 
chapter will now be discussed. The theorems presented give rise 
to a procedure which is optimal in deciding on the presence or 
absence of lines within a narrow rectangular region. 

For the theory to be discussed, we adapt the following 
from chapter lL: 

Let {I,(x, y)} be a set of real valued functions defined 
over a narrow rectangle R. 

Let D = {o, 1} be a two element description space. 

Let S, a sampling set, be a finite subset of R. 

Let F(I(R)) be an identification function mapping onto D, 
with the inverse image of the element 1 consisting of exactly the 
intensity functions given sige to by lines down the center of R. 

Let G(I;(S)) be a noise free sample identification function 
such that it and the function F(I(R)) conform to the descriptive 
completeness criterion. In this case, descriptive completeness 
requires that any line identified on R by F(I,(R)) be identifiable 
by G(I,;(S)) from intensities on S. 
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We shall make the following assumptions as well: 

1) That the intensity gathering device is subject to 
fundamental distortion, as defined in the previous chapter. 

In particular, we assume that this distortion governs the 
relationship between noisy samples J,(S), and corresponding 
noise free samples I.(S). 

2) That at most one line can occur in the set S. 

Some of these definitions and assumptions will not be explicitly 
used until the next chapter. In particular, the theorem about 
to be proved makes rather little explicit use of them. 

The optimization of the system in question will, as before, 
be a matter of making optimal use of a series of a priori and 
conditional probabilities in the minimization of certain error rates. 
In particular, we will assume that the a priori probabilities 
P(I;(S) of each noise-free sample (8S), together with the 
conditional probabilities P(J,(S)/1,(S)) are known; or alternatively 
that the joint probabilities P(Ji(S), 1.(S)), the product of the previous 
two, are known. Again we may wish to hold the true negative 
error rate to some value and minimize the false positive error rate; 
hold the false positive error rate to a certain value and minimize 
the true negative error rate; or minimize the sum of the two. 
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A surprising result emerges from the theorem about to be discussed: 
that these three types of optimization are brought about by 
essentially the same procedure. 

We need the following definition before stating the main 
theorem of this section. 
Definition: 


Q(3(S)) = DO PUL (S)/5(S)) (2.1) 


This quantity is the probability, given the noisy sample J(S), that 

there is a line down the center of the region R. 

Theorem: Suppose it is desired to keep the false positive error 
rate below a certain value a, and to minimize the true 
negative error rate. Then there exists a B such that 
the following decision function H(J(S)) satisfies the 
conditions: 

H(J(S))=1 <> Q(J(S)) 2B (2.2) 


Where B is the minimum value such that: 


D5 (1-Q(5,(S))P(S,(S)) < @. (2. 3) 
Q(I,(S))2B 
Proof: 1) Suppose a noisy pattern J(S) occurs, and that 


H(J(S))=1, i.e. a line is claimed to exist. Then 
(1-Q(J(S))) is the relative probability of a false 
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positive error whenever this pattern J(S) occurs, and 
(1-Q(J(S))P(J(S)) is the absolute probability of a false 
positive error involving the pattern J(S)). Thus: 


> -QU4(S))P(T,(S)) 
H(J4(S))=1 


is the over-all false positive error rate of the function 
H(J(S)). 

2) It is clear that the value 1 in place of 8 in 
equation (2.3) satisfies it, and that such values are 
bounded below by 0. Hence there exists a least value 
satisfying the equation, which we shall call £B. 

Also, in the continuous case, for this value of the 
equality of equation (2.3) will hold. One may 
consider this to be the case as well in the discrete 
case, if the number of discrete values is large. 

3) It remains to show that this procedure gives a 
minimum number of true negative errors. This error 
rate is given by: 


Q(J,(S))P(J;(S)) (2. 4) 
H(J4(S))=0 


Suppose we consider another decision function 
H*(J(S)) satisfying the condition of having false positive 
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error rate equal to or less than a, i.e. 


D> (1-Q(54(S))P(T,(S)) <4 (2.5) 
H*(J,(S))=1 


We wish to show that its true negative error rate 
is not less than that of H(J(S)), given by expression 
(2.4), i.e. that 


JAI (S)P(T US) > J) QT (SPITS) (2. 6) 
H¥*(J,(S))= 0 H(J,(S))=0 


4) Let R be the set of i's s.t. H(J;(S))=1, and 
H*(J,(S))=0, 
and let T be the set of i's s.t. H(J,(S))= 0, and 
H¥(J,(S))=1, 
and let U be the set of i's s.t. H(J;(S))= 0, and 
H¥*(J,(S))= 0, 
and let V be the set of i's s.t. H(J,(S))=1, and 
H*(J(S))=1. 
Then 


Y> QI, (S))P(T(S)) = 
H¥*(J,(S))= 0 


> AT USNPIT (SY) + DQ(ITUSP(T,(S)), 
U R 
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and 


das, (s))PU,(S)) = 
H(J,(S))=0 


> Q(5,(S))P(T,(S)) + @(g4(S)PUI,(S)). 
U T 


So to prove (2.6), we need to show: 


DAI USPT(S)) = DQUI,(S))P(I,(S)) (2. 7) 
R T 


If ieR, then Q(J,(S)) > B,and if ieT, Q(J(S)) < B, so 


>. Q(5,(S))P(F,(S)) 2 BD, PUS,(S)) (2.8) 
R R 

and 

>Q5,(S))PUT,(S)) < BY, PUS;(S)). (2. 9) 
T T 


Assuming equality in expression (2.3), we have 
from (2.3), (2.5) and the definition of R, T, and V: 


D (1-QU5,(S))P(F,(S)) = @ > UL -Q(54(S)P,(S)), 
RUV TUV 


or, expanding and eliminating the sum on V: 


> PU US)) - DQ(5,(S))P(T4(S)) 2 
R R 


DPSS) - DQI,(S))P(T,(S)). (2.10) 
af T 
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Combining (2.8), (2.9) and (2.10): 


1/8 DAI AS)P(T(S)) - Hacsy(s)P(s,(S)) > 
R R 

DPI AS) - D2Q(s;(S))P(T,(S)) > 

R R 

> PUS,(S)) - YQ(s,(S))P(T,S)) > 

T Tr 

1/B > Q(IAS))P(I,(S)) - }1Q(5,(S)) P(g, (S)), 
T T 


or: 


(1/B-1) )Q(5{(S)PUT4(S)) > (1/B-1) .Q(5,(S))P(T,(S)). 
R T 


Formula (2. 7) follows from the fact that (1/B-1) is 


positive. QED 


The case of minimizing the false positive error rate with 


a fixed true negative error rate is almost identical: 


Theorem: 


Proof: 


Suppose it is desired to keep the true negative error 
rate below a certain value a, and to minimize the 
false positive error rate. Then there exists a8 
such that the following decision function satisfies 
the conditions: 

H(J(S))=1 <=> Q(J(S))2 B . 


As for the previous theorem. 


It follows from these two theorems that whatever the value 
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of B used, the procedure of thresholding @(J,(S)) is better than 

all other procedures with the same false positive error rate, 

and better than all other procedures with the Same true negative 
error rate. In this respect it is Similar to a saddle point solution 
of classical 2 by 2 game theory. 

It is not difficult to see that this procedure can, fora 
suitable choice of B, be used to minimize the sum of the two 
error rates. Consider an arbitrary choice procedure H* which 
gives a false positive crror rate a and true negative error 
rate b. Then there exists a B such that thresholding Q@(J(S)) at 
B gives false positive error rate a and true negative error rate 
bs, where b* £ b. This will provide a decision procedure 
whose total error rate is a+ b¥ £ a+b. Hence thresholding 
of @(J(S)) with this B gives a minimal sum of error rates. 

Bie THE RELATIONSHIP BETWEEN GLOBAL AND 

REGIONAL LINE PREDICATES 

This section discusses the relation between the global 
line finding procedure described in chapter one, and the 
regional procedure of the present chapter. It is clear that the 
procedure of chapter one is computationally unfeasible. 
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However a systematic application of the regional predicate to a 
large number of regions on the field of view may be computationally 
practical. ‘The latter procedure, in certain -casés of interest, can 
not achieve the optimality of the global procedure. However it 
does trade a certain amount of accuracy for coinputational 
feasibility. 

It has been pointed out that in certain cases where lines 
on the visual field bear no systematic relationship to each other, 
a regional line predicate of the sort described in the definition 
at the beginning of this chapter secms to be potentially as 
accurate as a global one. It is tempting to formalize this notion 
into Some sort of theorem, stating conditions under which a 
systceinatic application of a regional predicate of this sort to a 
suitably large number of regions on the field is equivalent to a 
global procedure. Unfortunately, an exact correspondence 
between these two procedures is difficult to make. One problem 
is illustrated in Figure 2.3. It is reasonable to assume that in 
the system described in chapter one, if an clement of the 
description space contains a segment AB, it would not also 
contain a subsegment ol that line. More informally stated, the 
scene descriptions would not contain redundant segments. 
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Z A REGIONAL LINE PREDICATE 
APPLIED TO MANY REGIONS ON 
THE FIELD WOULD REPORT LINES 
AB, AC, ..., AZ, BC, ..., BZ, ETC. 
A GLOBAL PROCEDURE WOULD 
D OMIT THE SUB-SEGMENTS AND 
C REPORT ONLY AZ. 


Figure 2.3 
On the other hand, the systematic application of a regional 
predicate of the sort mentioned would not only give positive 
identification for some complete line in the scene, but for all 
examined subsegments as well. Another difficulty is that the 
regional predicate might report the existence of two nearly 
coincident lines at very slightly different inclinations, where 
there is actually only one line. These difficulties will be 
encountered in chapter four, which describes an actual computer 
program embodying a regional line predicate. In the program, 
the problems are handled at the heuristic level. However, from 
a formal point of view, obvious efforts to solve them appear 
artificial. We will thus confine the remainder of this discussion 
to the informal level. 
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It is convenient to informally divide the evidence relating 
to the existence of a particular line into three categories: 

1) Global evidence, namely evidence from areas spatially 

remote from a possible line; 

2) Regional evidence, namely evidence from an area 

within a small radius of a possible line; 

3) Local evidence, namely evidence contained within a 

neighborhood of some point on a possible line. 

It has been pointed out that there are certain problems 
associated with a formal identification of a systematic use of a 
regional predicate with a global predicate. This identification 
appeared plausible only in cases where there is no relationship 
among the lines in the figures. However, the situation of 
particular niSeest is that in which the scenes consist of sets of 
plane-faced solids. In this case there is definitely a relationship 
among the lines in the figures; and a regional predicate cannot 
possibly achieve the optimality of a global one. Figures 2.4 
and 2.5 illustrate the possible failures of a regional line 
predicate, which might be avoided by the use of global evidence 
as well as regional evidence. 
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plausibility to closed line figures. Figure 2.5 illustrates a 


9 
DISTINCT 
LINES 


FAINT 
LINE 


THE REGIONAL PREDICATE 
APPLIED TO "A" MIGHT CLAIM 
THAT THERE IS NO LINE. 
HOWEVER, GLOBAL EVIDENCE 
STRONGLY SUGGESTS THERE 
IS. 


Figure 2.5 

situation similar to that in Figure 2.4. In this case a line is 
totally ignored by the regional predicate, despite considerable 
global evidence for its existence. 

In chapter four, a program will be described which uses 
a regional predicate in the analysis of scenes consisting of 
prismatic solids. It is convenient to here describe briefly two 
features of this program, as the current context justifies their 
use. Specifically, the two features are designed to overcome 
the non-globality of the regional predicate as illustrated by 
the previous two examples. The sort of problem illustrated in 
Figure 2.4 is overcome by the use of what will be termed links, 
which tentatively extend a line to a possible vertex. The sort of 
problem illustrated in Figure 2.5 is handled by the procedure of 
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proposing lines. The procedure in effect assigns a high a priori 
probability to lines suggested by lines already identified. This 
tends to augment regional evidence for the proposed line. 

A final fact should be pointed out in this context concerning 
the relationship between the use of purely local evidence and the 
use of a regional predicate. This is analogous to the relationship 
between the systematic use of a regional line finder and the use 
of a global one. In the latter case, it is necessary that the lines 
in the scenes be spatially independent in order that the two 
procedures be equally effective. In order for purely local decisions 
as to the existence of a line to be as effective as regional decisions, 
it would, by analogy, be necessary that the points along a particular 
line be spatially independent. This is, of course, totally contrary 
to the nature of aline. It is thus quite impossible that a 
systematic application of a simple local decision procedure to a 
suitably large number of neighborhoods covering a region could 
be as sensitive in detecting lines as a regional predicate. By a 
"local decision procedure" is meant a procedure which decides 
entirely on the basis of intensity information from within a 
neighborhood of a point, whether or not a line traverses the 
neighborhood. The output of such a procedure would be a set of 
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points on the visual field, which would be incorporated into lines 
by finding maximal sets of adjacent colinear points. 

This two stage procedure is a special case of a set of 
procedures which process intensities in a neighborhood, and then 
process the results for a set of neighborhoods contained within a 
region. Aithough the particular two stage procedure mentioned 
in the previous paragraph can never be as sensitive as an optimal 
regional line finder, there exist two stage procedures of this sort 
which are. This is not surprising, Since all regional predicates 
fall into this class, if one regards the first stage to be the 
identity function, and the second stage to be the regional predicate 
itself. There also exist non-trivial examples of members of this 
class which are as sensitive as an optimal regional procedure. 
The procedure described in the next two chapters involves such 
a two stage predicate. For the first stage, four values are 
obtained from the intensities in a neighborhood. Sets of these 
four-tuples, obtained from all the neighborhoods within a region, 
are processed by the second stage; and a decision is made as to 
the existence of a line in the region. The procedure may be 
regarded intuitively as extracting all necessary local evidence in 
the first stage, and employing all necessary regional evidence 


in the second stage. 
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CHAPTEIER 3 


COMPUTATIONAL APPROACHES TO THE REGIONAL THEORY 
De INTRODUCTION 


The object of this chapter is to give a computationally 
feasible method for computing values of the function given by 
Formula 2.1 of the last chapter: 


Q(J(S)) = DY PUL(S)/I(S)). 
1 


It will be recalled that this is a function of the finite set J(S) 
of intensities obtained by a particular sort of optical input 
device from within some region on the field of view. ‘The 
thresholding of this value provides an optimal regional line 
detection procedure. 

From Bayes!’ rule it follows that 


POS) 1(S)) 
P(J(S)) 


P(L(S)/S)) = 


30 


so that 


P(J(S)) 
Using the relation 


P(I(S)) = D* PUI(S), 1,(8)) + 2 PCI(S), 14S). 
G(L;(S))=1 G(I;(S))= 0 


and 

P(J(S), L(S)) = PCI(S)/1(S))PU CS), 
we have: 

@Q(J(S)) = 


>, P((S))/1(SPU(S)) 


> P(MS)/1(S8))PU(S)) + > P(I(S), L.(S))PC,(S)) 
G(L,(S))=1 G(I,(S))=0 


sections IT and II] of this chapter will be concerned with 
the calculation of the values of P(d(S)/E,(S)). Sections IV and V 
propose a model for the relative probabilities of the various 
noise-free patterns 1,(S). On the basis of the values of 
P(I,(S)) given by the model, it is possible to express the sums 
in the above expression as integrals, and to give a relatively 
simple explicit formula for Q(J(S)). In the last section of this 
chapter, this formula is applied to some simple cases of 


image-dissector output. 
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II. NOISE-FREE PATTERNS AND BLURRING DISTORTION 


The value of the conditional probability P(J;/1;) isa 
function of the relation between the two sets of intensities 
J; and ie This relationship can be decomposed into: 

1) a determinate relation between the noise-free 

sample I; and a distorted but noise-free version . and 

2) a simple probabilistic relation between this : and 

the noisy pattern J;. 

The former will be discussed in this section; the latter in the 
next section. 

According to the definition of fundamental distortion, 
the relation between a noise-free pattern I, and a distorted 
version I* may be expressed in terms of a convolution of the 
former with some function f(x, y). Possible examples of 


f(x, y) are given in Figure 3.1. Theoretically, from the way an 


HILL FUNCTION PILLBOX FUNCTION 


Figure 3.1 
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image-dissector is designed, this function should be cylindrically 
Shaped as in the right hand illustration of Figure 3.1. In fact, 

it is more like a gaussian "hill'' as shown in the left hand 
illustration. An intensity function I(x, y) is acted upon by this 
convolution function f(x, y) to produce a blurred intensity 


function I*(x, y) as follows: 


cO co 
I*(x, y) = J fio, y-v)du dv. 


— 00-00 


If we had an edge whichis reasonably uniform for a 
relatively long distance compared with the ''width"' of the 
convolution function, we can describe the convolution effect, at 
least locally, by means of a single integral instead of a double 
integral. In particular, suppose for some local region the 
intensity function I(x, y) represents a line parallel to the y-axis. 
Then locally we may write I(x, y) = I(x) and the corresponding 
blurred but noise-free intensity function I*(x, y) may be 


expressed in this region as: 


00 00 
I*(x, y) = ff fr. o1t-aau dv 


- fru fie. rav du. 
-00 -00 
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Since the inner integral is a function only of u, we may 


indicate it with a single function name: 


fee) 


g(u) = fis oev, 


-00 
and we have at a reasonably extensive and uniform edge 
parallel to the y-axis: 


i.) 


I*(x, y) = I*(x) =f te-wetwas. (3.1) 


-@ 


We may obtain g(u) empirically by scanning at right angles 


across a very narrow line in the visual field. Let the intensity 


along a line normal to a vertical line in the visual field be 


given by: 
I(x) = b if x <-a, x>da 
I(x) = c -ac x¢a. 


Then the distorted version of the intensity locally around the 


line is given by: 


0 -xta 
1*(x)  fre-ugarae = © fecuran + b. 
—o0 -x-a 
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-xta 


But Lim © fatwran = c-a-g(-x), so 
a>0 
-x-a 
IX(x) = ceag(-x) + d, 


for very narrow lines. 


Figure 3.2 shows plots of intensity taken along normals 


INTENSITY PROFILES 0 


(NEGATED) 
NORMALIZED TO HAVE 
SIMILAR AMPLITUDES 


1 UNIT = 1/2000 VEDI FIELD 


Figure 3.2 
to various vertical lines. According to the preceding 
paragraph, these curves should be of the form a-g(x) + b. 
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They are shown superimposed and normalized to have similar 
amplitudes. The lines are all quite narrow relative to the 
width of the hill plotted, and represent lines of a wide variety 
of intensities. The values are all proportional to absolute 
intensity, having been suitably transformed from vidissector 
output which is logarithmic. The degree of variation of 
un-normalized amplitude was greater than ten to one; and may 
be observed from the relative noisiness of the curves. All 
were recorded at a fixed noise level, and the amount of local 
fluctuation of a given line in the illustration is a measure of 

the degree to which the hill in question was enlarged in the 
normalization process. It should be noted that the shape of 

the curves is remarkably consistent. This indicates a good 
agreement between the theory just discussed and the actual 
behavior of the blurring process in the vidissector. Figure 3.3 
illustrates a curve obtained from the data in Figure 3.2 by 

hand fitting. Again, this curve should be of the form 

a-g(x) + b, where g(x) is the one-dimensional blurring function. 
Also illustrated is a curve of the integral of this function, 
obtained by hand integration. This latter curve will be 
compared with a similar curve obtained by two different methods 
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15 

HAND FITTED CURVE 
OBTAINED FROM 
CURVES IN FIGURE 3. 2 
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INTEGRAL FROM 0 TO X 
OF THE ABOVE CURVE, 
OBTAINED BY HAND 

INTEGRATION 
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APPROXIMATIONS TO g(x) AND 
ITS INTEGRAL MAY BE OBTAINED 
EMPIRICALLY FROM THE DATA 
IN FIGURE 3.2 


Figure 3.3 
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which will now be described. 
Another approach to finding g(x) is based on the fact 
that the peak value of an intensity scan normal to a line of 


non-negligible width a is related in a Simple way to the integral 


a/2 


fe dx, 


~a/2 
namely, if the actual intensity along a normal to a narrow 
vertical line is given by: 


I(x) = ¢c if x<b, x>bta 


I(x) ct+d b< x¢ bra, 


then I*(x), the distorted intensity function is given by: 


ce oO 


I*(x) =f ro-wearen = foc ae I, (x-u))g(u)du, 
-00 


~-0 


where I (x) = 0 except in the interval (b,bt+a), where 


I (x) = d. One may write the latter integral as: 


io) 


lo] 
ef etwas “f I, (x-u)g(u)du, 


— 00 —oo 


But the former integral is a constant, and the latter integral 
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is equal to: 


bta-x 


a fetaran, 


b-x 
and has an extremum at x = 1/2(a+b), which is a minimum or 
a maximum depending on the sign of d. At this extremum, the 
integral has the value: 
a/2 


a fear 
-a/2 


Thus a scan made perpendicular to a line of width a and 
intensity ctd on a background of intensity c has a maximum 
(or minimum) whose value relative to the intensity somewhat 


away from the line is: 


a/2 


a feu. 


-a/2 
Suppose a scan is taken across a pair of narrow 
triangles as in the left of Figure 3.4. In the figure, 
kLn(1/intensity) is plotted for successive scans. In the right 
hand side of the figure, the values of the maxima of each 
scan minus the background value are plotted for each scan as 
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Figure 3.4 
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a function of the spatial location of the scan along the set of 
triangles. Since the width of the triangle at a particular scan 
is a linear function of the position of the scan along the 
triangle, the plot just mentioned may be considered to be a 
function of the width of the line being scanned. Hence the 
x-axis in the plot is in units commensurate with this distance 
measure, and in the same scale as is used in Figures 3.2 
and 3.3 relative to the physical dimensions of the vidissector. 
The values plotted are thus those of kLn(1/m{(x)) - kLn(1/c), 
where m(x) is the actual minimum (since the target is dark) 
of intensity along a scan of a line of width x, and c is the 
background intensity. If the blurring function behaves 


according to the theory proposed, we should have: 


x/2 x/2 
c ~ m(x) = a f etuan = 2d fetura (3. 2) 
-x/2 0 


where d is the absolute value of the intensity difference between 
the background and the target. Using the curve of the integral 
of the function in Figure 3.3, and doing a certain amount of 
arithmetic involving logarithms, we can obtain a curve of 


predicted maxima in accordance with the premise expressed 
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in (3.2), and with the assertion that the curves in Figure 3.3 
represent g(x) and its integral. This curve is indicated in 
Figure 3.4 as the dashed line superimposed on the plotted 
curve in the figure at the right. The closeness of these two 
curves indicates good consistency between these two approaches 
to finding g(x); and provides further evidence that the curves in 
Figure 3.3 represent g(x) and its integral. 

A third approach to the computation of g(x) is based on 
the fact that at a wide edge, the value of the intensity along a 
line normal to the edge is related in a simple way to the 


function: 


hd 


h(y) = Af eorex. 


—-@ 


In particular, if the intensity function for a vertical edge is 
given by: 

I(x) =a x< 0 

I(x) = b x>0, 


then the distorted value of intensity, I*(x), is given by: 


io.¢) oo 


I*(x) = freweturan =at b feturan. 


—C -xX 
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This equality is derived by an argument Similar to that given 
above. In Figure 3.5 we have the values of kLn(1/intensity) 


o POINTS OBTAINED FROM THE 
INTEGRAL CURVE IN FIGURE 3.3 


Oe SCAN 
LINE 


INTENSITY PROFILE AT AN EDGE SOURCE 
1 UNIT = 1/2000 VIDI FIELD WIDTII OBJECT 


Figure 3.5 
for the intensities along a line normal to an edge, subject to 
the blurring of the vidissector. The illustrated points along 
the curve represent valucs obtained from the integral curve 
in Figure 3.3, suitably scaled.. For clarity, the curve 
represented by these points has been omitted. The close 
correspondence between the points and the curve further 
verifies that the curves of Figure 3.3 are a good approximation 


to g(x) and its integral. 
III. NOISE AND THE COMPUTATION OF P(S;{/T) 


A given noise-free sample I; turns into a distorted 
sample us as shown in the previous section by the convolution 
of iF with some blurring function. ‘The conditional probability 
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P(J;/1;) may easily be expressed as the probability that simple 
noise on the sample Ux would give rise to the configuration of 
intensities Jj. 

It has been claimed that a large number of intensities 
read at a given point under a given lighting situation by the 
vidissector have approximately a normal distribution. 
According to the way in which the vidissector is constructed, 
the absolute noise level is a constant for the logarithmically 
scaled intensities delivered. Actually there are four possible 
noise levels; however all work in the remainder of this paper 
will involve only one, the most accurate. Hence we shall 
assume that there is a single constant noise level. The 
intensities read at a particular point for a particular set of 
lighting conditions follow the distribution: 

ae. 

wa e abd a, 
where ais aconstant. From this may be computed the 
probability of a given configuration of intensities 
Wisp Sag ae at points with actual intensities 


La Since the noise-induced deviations are 
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statistically independent, this probability is: 


m _l [- =V¥i- ||? 
1 2 a | 
aan dvydvy...dvim, 


a 2 
Di ij ae 2 (u,-v, ) ’ (3. 3) 
with : 
Ik = (u,, Uy ,u_) 
we (vy, Vo; Vin) 
we have: 


1 B 
a’ (3. 4) 


= xy = pee eee 2 
P(S;/T;) = P(S,/TP (avany e dv,dvg...dvp. 


The value of a for the signal to noise ratio has been reported 
(Horn 1969) in terms of the percentage error of a logarithmic 
intensity value corresponding to an error of one standard 
deviation of the errors at that intensity. Again, this 
percentage is a constant for all intensities, and is reported to 
have the value 1.6%. Since the logarithmic intensities are 
given by: 

100 Log, (1/1), 


8 


the absolute error corresponding to one standard deviation 
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is: 

a= 64. Log, (I- 0161) - 64. Log, (1) = 1.5 (3.5) 
This is the value of a which will be used in (3. 4). 

The procedure for computing the conditional probability 
P(T(S)/T(S)) is to determine the blurred version I* (S) of the 
set of intensities L(s), and to calculate the value of 
P(S(S)/TE(S) by (3.4). The relationship between I,(S) and 
Ik(S) is in general governed by the function f(u, v) referred to 
in the previous section. However we shall consider only 
cases where (3.1) applies, and will need only the function 
e(x) to determine this relationship. 

TV -« A MODEL FOR TILE Si'E nea AND COMPUTATIONAL 

CONSEQUENCES 

In this section we shall give models for the sets 
{Is (S)} of noise-free distorted intensity configurations taken 
over particular rectangular regions S. Our attention will be 
confined to sampling regions 5 which have a particular width, 
and whose lengths are integral multiples n of the width. We 
may consider such a region to be composed of n Square 
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sub-components, denote it by S,2 and use the definition: 

1e(S_) = TY. 
Rules will be given for assigning a priori probability values 
P(T;) to the elements of {1«;}. It will be recalled that there 
is a determinate 1:1 correspondence between elements IK; 
and elements I(S,). Thus the analysis in section I of this 
chapter remains valid with I%; in place of 1,(S). It follows that 
the value of Q(J(S)) of (2.1) is determined by (3. 4) and by the 
values of P(I*;'). The latter are the subject of this section. 

Consider, for a fixed n, the set {iF} of all noise-free 
distorted samples over rectangular regions composed of n 
Square sub-components. We shall divide this set into four 
subsets as follows: 

Let CL be the subset of {1*;} consisting of sets of 

intensities from lines in the visual field exactly 

centered in the rectangular region. 

Let CE be the subset of {1*;} consisting of sets of 

intensities from edges in the visual field exactly 

centered in the rectangular region. 

Let CH be the subset of {rs} consisting of sets of 
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intensities from a homogeneous area in the rectangular 
region. 

Let CS be the subset of {re} consisting of sets of 
intensities from lines or edges which are not centered 
in the rectangular region. 


We Shall first consider a model of CL, and base it on 


the following assumptions: 


1) That the absolute intensities of the samples of CL 
are of no consequence, so that in each square 
sub-component of a sample ie the sum of the intensities 
may be assumed to be 0. 

2) That on any section across some j-th sub-component 
of a member 1k; of CL, the intensity function has the 
form: 


where By | will be termed the relative amplitude, and 


J 


b. , is chosen to conform with assumption 1). 


> 


3) That the relative amplitude a; , of an intensity peak 


J 
(or valley) is uniform within the j-th sub-component of 


I*; A sample Te, may thus be described in terms of 
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an n-tuple of relative amplitudes (a; porte ay ads e. 2. 

as Tea, ee as ale 

4) That each sample 1x? was given rise to from some 

line, highlight, etc., in the real world with an 

"idealized" relative peak (or valley) amplitude ai, 

which is uniform along the length of the line. 

5) That this idealized relative amplitude is purturbed 

randomly along the length of the line so that there is a 

minor fluctuation of relative amplitude aj; among the 

various sub-components of a sample Tks with a 

normal distribution with variance a. 

The first of these assumptions ignores a certain amount 
of information about a line which is useful in distinguishing it 
from anon-line. Broadly speaking, it prevents a distinction 
between a series of peaks in a line whose absolute intensity is 
constant or regularly varying; and a series of peaks in a line 
whose absolute intensities vary randomly or discontinuously. 
This is not undesirable in the case of a line lying in the real 
world partly in shadow. However, it would be desirable to 
eliminate as lines a series of ''lined up" intensity peaks whose 


absolute intensity varied randomly, as this would likely be due 


69 


to noise. 

The second assumption is justified on the basis of the 
discussion in the first section, and by the fact that we are 
considering noise-free but distorted samples. 

The third assumption is justified on the basis that the 
"ridges" of intensity of samples in CL are rather uniform 
along their whole lengths, and would thus be quite uniform on 
square sub-components. This assumption is particularly valid 
if a square sub-component is only sampled along a line 
traversing it perpendicular to the orientation of the rectangle. 
This is approximately what is done in the program which will 
be discussed in the next chapter. 

The "idealized'' constant relative amplitude of 
assumption 4) may be thought of as the relative amplitude 
which would be obtained in the absence of physical defects of 
the object being viewed or on the vidissector photocathode, and 
in the absence of Spurious local reflections and anomalies of 
lighting. The purturbations of assumption 5) may be thought 
of as a result of these defects etc., e.g. a nick on the edge of 
a cube, a slight shadow or a bad Spot on the photocathode of 
the vidissector. ‘lhe idea of an idealized constant value of 


70 


the relative amplitude along an edge is further justified by the 
fact that intensities used will be logarithmic. Any ridge of 
intensity, for example a highlight, is a relative contrast 
phenomenon, and may vary in absolute intensity, but not in 
relative intensity. A logarithmic intensity function would tend 
to register amplitude of highlight the same regardless of the 
absolute intensity. The same is true when the argument is 
applied to intensity discontinuities at an edge. 

It follows from these assumptions that if we let F(a) 
be the a priori probability of an edge in the real world with 
"idealized'' intensity a, then the probability of existence of 
a particular I*; = T¥(a; porte Ay n) given rise to by an edge 
with idealized relative amplitude a and parameterized by 


amplitudes Byes? An is given by: 


Twig ee 
1 2L 9, 
F(a) “owam é da; ee daj n- 
j=l 


The probability that this TK, would occur from a source with 
any idealized amplitude is given by the sum over a of the above 
expression. This sum is simply the a priori probability P(I*; ) 5 
P(T*(a; Lecre se A n* 

This sum may then be evaluated provided the distribution 
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F(a) is known. Assuming it is normal with standard 


deviation p.» we have: 


vn = Ba. @ AL 


Pay 2 7 
where P(CL) is the a priori probability of existence of 


amember of CL. We then have: 


P(THlay ys +21 5 8%, p))= 


aa Eoin e? laf al ee dada 


yor"™"'o"o, Was oes 


This sum, including the differential da may be expressed as 


as integral, with the exponent slightly rewritten: 


na - 2adaj + ne 2a pen + Feu] 
few rr da. 


The exponent may be further rewritten as: 


2 
Letting: 
1 n 
A or tr 
2 aij 
= on 


and completeing the square, the exponent becomes, in 
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n(3- 6) 


terms of A and B: 
-4 [ a(a?-2a(B/A) + (B/A)*) - BY/A + Eai/oi], 
i 
The integral becomes: 


fe -A/2(a - B/A)’ - 1/2(¥a? /o2- B?/ A) 
i as. 
-0 
The second term in the exponent is independent of the 
integration variable a, and may be taken outside the 
integral sign. The remainimg definite integral is well 


known to have the valuey7/A. Thus the integral is 


equivalent to: 


1 eS al 
7 ola A 
yze : 


The result of substituting in this expression the values 
of A and B, of substituting the resulting expression into 
(3.6), and of doing some straightforward algebraic mainpulation 
is: 
PUT(a, iF )= 


_ | z ai, npa ¥(aij- 3)” 
P(CL) 9% 2[0%+np%  o02( 02+ np) 
Vinon Varrnpt © sie acne al 


73 


An analysis of the CE category may be made in a nearly 
identical fashion. In this case, the function g(x) of assumption 2) 
is replaced with the function h(x), its integral. As was pointed 
out in Section II, this function gives the intensity along a normal 
to an edge, and is the obvious correspond to g(x), which gives 
the intensity along a normal to aline. A formula for the 
a priori probability of a member of class CL may be derived 
from an analysis nearly identical to that given above. The 
resulting formula differs from (3.7) only in that different values 
appear for the variances g, and g,; and that P(CE) is used in 
place of P(CL). 

Again, an analysis of the CH category may be made along 
nearly identical lines. In this case, the function g(x) is replaced 
with l(x) = ax + b, the function which gives the intensity profile 
across a homogeneous region. Again the resulting formula for 
a priori probabilities of members of CH differs from (3. 7) only 
in the values of o, and p,; and in the use of P(CH) in place of 
P(CL). 

At this point it is convenient to recapitulate some of the 
foregoing by rewriting the formula for Q(J(S)) appearing at the 
end of Section I in terms of the specific notation introduced in 
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later Sections, and some additional new notation. First, it will 


be recalled that the notation Pe(a, » ay a) was introduced 


L? 
to refer to noise-free distorted Samples over a region consisting 
of n Square sub-components, and containing a line centered in 
the region. A value oe refers to the relative amplitude of the 
intensity profile of the i-th sample, taken across the j-th 
sub-component. We may similarly adopt the notation 

Tete: ett be ad to refer to noise-free distorted samples 


from the class CE, and [*(c. 


Laps Soe OCs n) to refer to samples 


from CH. We shall reserve the notation Ix. to refer only to 


n 
i 
distorted noise-free samples from the class CS. We may 
introduce Similar notation for noisy samples by a definition 
Similar to that used at the beginning of Section IV: 

se on See 
By this notation is meant the i-th possible noisy sample ona 
region consisting of n Square sub-components. 

The sets CL, CE, CH and CS provide us with an 
alternative for expressing the function G(1,(S)) used in the 


formula for Q(J(S)) in Section I. If we make the following 
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definitions: 


CLAI;") = 

n ale 
a: eR Ce nd)s (3-8) 
CE(J;) = 


n Ie ale 
ee vee 4 By n) PUM: see by ah (3.9) 


CH(J;) = 

DI PW, THe, ys 6s 2 Oy (PU, 1. ess Cy yd) (3.10) 

1 

Cs(g}) = DPR Pp PF). (3.11) 
1 


Then it follows from the 1:1 correspondence of distorted 


noise-free elements, and non-distorted noise-free elements 


that: 


and 


n = n n 
D PUTS, WPUS,)) = CLIT,) + CEC), 
GAL (S,))=1 


n = n a 
2 PU; /T AS, PU (S,)) = CHF) + CS(S;). 
G(L,(S_,))=0 


Consequently we may rewrite the formula at the end of Section I 


as; 


j=} 


CL(J}) + CE(J:) 
(3. 12) 


jen} bee 


n 
Se ne ee ee ED 
a CLS; fe CES; )+ CH(J.) + cs(I,) 


ow 


76 


We may now turn to an analysis of the class CS. It turns 
out not to be necessary to make an explicit model as was done 
for the classes CE, CL and CH. To See this, recall 
Formula (3.12) of the previous paragraph, and Formulas 
(3. 8) - (3.11), which define its components. An element in 
these latter sums, e.g., 


P(S;/ Te(a, 


ses og: Bs 
2,0? eset 


MPU a, ys ees 8 OD) 


’ 


from the sum CL, may be thought of as a measure of 
Similarity between J; and the distorted noise-free sample 


I¥*(a, 


idee ey, n): Weighted by the plausibility of the latter. 


Thus the value of CS, for example, is roughly proportional to 
the plausibility or frequency of occurrence of members of CS, 


n 
and to the similarity of Jj to members of CS. 


n 
J 


which is centered on the image of aline, the value of 


If J; occurs over a rectangle S, on the visual field 
CS(J;) is quite small relative to the other members of the 
denominator of Qs), owing to the relative infrequency of 
members of CS as compared with members of CH, and to the 
fact that such a a is as similar to a pattern of intensities 


from a homogeneous region as it is to a pattern of intensities 
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containing a skewed line. Since the value of CSU; ) is thus 
small, it could be omitted in this case, or at least approximated 
with the sum of values of 

Cn) + CEM) 
over a set of regions s* which are nearly coincident with S_, 
but are slightly skewed with respect to it. These values may be 
obtained "for free" for any region S,, by a line-finder which has 
to be able to find lines in all positions in the visual field; since 
it would necessarily compute Cig) and ces) in the 
process of testing the other regions = for the presence of lines. 

If 5, is a region in the visual field which did not contain 
aline, a noisy sample J, over 3 would, by a similar argument, 
have the property that CS(I;) makes a rather small contribution 
to the denominator of QUT}, and could be similarly approximated, 
or indeed omitted. 

It is only for the case that the region 5, contains the 
image of a skew line that the value of CSI") becomes 
significant. However, this effect occurs only for rectangles 
approximately coincident with a rectangle 5,, centered on a line 
in the visual field. Let Q*(J7) be the Same as Qs) but with 
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the value of CS(J;) omitted from the denominator: 


CLIT?) x CE(J;") 


CL(J;) + CE(J.) + CH(;) 


It has been empirically checked that Qx(J,) obtains a maximum 
value locally approximately on the rectangle S,, centered on the 
line. This is quite plausible, since the value of Quy,’ should 
be large for rectangles approximately coincident with Sy and 
if it didn't have a maximum on Sy it would either be offset 
parallel to S,, which is unlikely because of the symmetry of 
the intensity pattern; or there would be several maxima, 
which seems not to happen. Hence the omission of csts;) 
from the denominator of QUs;) has the effect of giving 
erroneously high values on regions nearly centered on a true 
line, but not erroneously high values on the maximum value 
region, namely a region centered on the line. However, these 
values are not so augmented as to create false maxima for 
regions not quite centered on the line. 

In summary, the set of noise-free distorted samples 
is partitioned into four classes CL, CH, CE and CS. Our 


threshold function Q(5;) is expressed, in (3.12), in terms of 
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four sums CLAS), CH”), CEU;) and cs(J;) over these four 
classes. Explicit models of CL, CH and CE allow a simple 
explicit expression of the a priori probabilities of members 
of these sets. These probabilities, together with the results 
of Sections II and III, determine the values of the sums 
CIS), CHU, ) and CEJ, ). It has been shown that C8(,) 
may be computed indirectly, or even omitted from the 
denominator of QU). The latter is permissible, resulting in 
Formula (3.13), in the case where the regional predicate is 
eooiied systematically over the whole field of view. 
V. COMPUTATIONS OF CL(J-), CE(J;") AND CH(J?) BY 

MEANS OF INTEGRALS 

In the previous section, it was pointed out that the 
threshold function Qts;) could be approximated with the 
function QJ; given by (3.13). This function is determined 
entirely by the values of sums CLIS;), CE(I;) and CH(I?), as 
defined by (3.8), (3.9) and (3.10). In this section, we shall 
show how to express these sums as integrals, and to reduce 
them to relatively simple closed form. 

It might seem a good strategy to evaluate the 
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expression, for example CL(S;) as given by (3. 8), by 
substituting values given by (3. 4) and (3. 7) respectively for 
the two terms within the summation sign, expressing the 
resulting sum as an integral, and finding a closed form for 
the value of the integral. This approach appears to run into 
difficulties, and a slightly different approach will be used 
instead. 

Our first step is to express, e.g., CL(J?) in a more 
convenient form. Since the sum in (3. 8) is taken over all 


n-tuples, CLS?) may be expressed as: 


= 


i,1 i,2 
n 
e2 PCS; [Te(a, iat AOS, Be AG n)) PUM ay ys sees “i n))s 
a. 
ign, 


where the Sum over a; , means that ak in the Summand 
takes on all positive integral values. We may express 
PUT#(a, pete s Ay n)) as 


DPUHMay ys sees a y)/a)P(a), 
a ' 


where P(a) is the a priori probability of a line with 


idealized amplitude a; and P(I*(a. 


Leo? a; )/a) is the 


relative probability of occurrence of a noise-free distorted 
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sample with sub region amplitudes ay a , given 


1? 1,n 
a line with idealized amplitude a. The conditional probability 
P(T*(a; fees 5 By n/a) may be expressed, as in Section IV, 


as a product of probabilities: 


n 
P(P(a; Lette ai n)/ 2) ~ Siac fa), 


where Pla, i) is understood as the probability, given a line 
with idealized amplitude a, that the k-th sub-component of 

a noise-free sample will have a relative amplitude k ‘Ene 
above probability may be expressed as a product, Since the 
elements in the product are statistically independent. 


Likewise, by statistical independence, PUT D(a; ae ae 


may be expressed as a product: 


n 
sk ss aes 
ee pastes yey [ Jew, (Kay yd). 


where Jk) is understood to mean the set of intensities of the 


noisy sample J; which lie within the k-th square sub-component 


of the region. Substituting and rearranging, we have: 


CL(J}) = 

iF : 
DSPay HY [Jeu oo/a, pra, ,/a) 
a a a k=] 
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It is not very difficult to see that the product over k and the 
Summations over the a; 8 may be interchanged, so that: 


CLUS,") : 


pa Pla) [1S PI(K/a, Play fad. (3. 14) 


Neti 
. n n 
Similar expressions exist for CE.) and CHa \s 


An expression for PUT MK)/ a, 


i, » may be obtained by 


adapting (3.4) to the present context, using the value of a 


from (3.5): 


cl rae 
n eo) m 
PS teen oS ee di eee i (3.5) 
J i, k san)" e° j,k Ja 
where 
i 
2 
2.4, ye (3. 16) 
Mi j,k vik 


which is similar to (3.3) in that it refers to some i-th noise- 
free distorted sample and a j-th noisy sample, but has a 
third subscript to indicate it involves only the k-th 

t ; ’ 4 
Sub-component. Tne values u; , are the m intensity values 

; ks 
within the k-th sub-component of the i-th noise-free distorted 
sample containing a line; and are taken over points which may 
U i : : : 

be denoted by p. .. Since the amplitude in this region is 


lk 
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as we We have by assumption 2) of Section IV that the intensities 
at points Pi, k are given by: 

UF ic = 4, KECE, ke) + BG, Ke 
where xf, k is the perpendicular distance between Pi, ke anda 
line traversing the center of the region in the direction of the 
expected line. If the geometry of the points PE ; iS fixed for 
all samples and sub-regions, then the value of glxy ;) is 
independent of the sample and sub region, and we may 
express such an intensity value by o(xt), By assumption 1) 
of Section IV, we may choose Dik so that the sum of uf, ; is 
zero, i.e., by choosing Dk = g. Hence defining values 


Cg em: 6 tae | 


ut = a(x*) - E alx')/m, (3. 17) 


by: 


we may express the m intensities in the k-th sub-component 


of the i-th distorted noise-free sample by: 


(uj, Ks » Ui, k) = 
(Bia, vey. AERO) (3. 18) 
where 
be 
2, U =i: (3.19) 
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To be consistent with assumption 1), we should also assume 


that: 


= Vik = 0 (3. 20) 


The value of D; j,k of (3.15) may be expressed in terms of the 
values just defined as: 


: t ty2 
Di i,k = 25, ay iO) (3. 21) 


It is easy to show that: 


Di, j,k = ak - aE IR, (3. 22) 
where: 
» ure ik 
ax - JoK (3. 23) 


b = Y'y? (3. 24) 
T 

R. eT. x uy? (3. 25) 

kod V5, yok ‘ . 


Combining we have, omitting the differentials avi k: 


ke ‘ ar)” + Rix 
(1.5)? (3 


n 
PUT, (e)/ ay . 26) 


1 eo 
) = —____ 
Kk’ (1. 5y27 )” eC 
We may express Pla; / 2) as before by assuming a 
normal distribution on the various values of a; y.. With 
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standard deviation go and mean a: 
n — 


1 - - “*] 
1 a oO 
‘ ee n Gas =. (3.27) 
Pla, | /a) ee i,k 
Combining (3.14), (3.26) and (3.27) we have, again omitting 
differentials oy KS 
n 
n 
CLIT) = >oP(a) : 
: a U2 aeaaer 
i, k 
Li] bax - ary ie + Ri, (a - ax )? 
Bos - ——____—_ 
C-s ey os da,,da (3. 28) 
a; da (3. 
The 


with b, a i: and R. k given by (3.23), (3.24) and (3. 25). 


inner sum may be put in the form: 


which admits of the approximation used in the preceding 


section: D 
-Bx” + Cx + D 
DN 
Ce 
—-+) 
4B 


= 2 
-Bx + Cx-+D — 
f vin 
A | 4 dx = AN® e 
VB 
—o 


Applying this approximation to the inner sum in (3.28), we 
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have after considerable algebra: 


i 1.5 
Ty me 
cir ati Leal | (yam 1. SY. 5) be: 


_1] Rix , Plate ~ a)? 


e Ace (1.5) + be? Jee 


t 
where again, the diffentials dv. have been omitted. 


j,k 
may alternatively be expressed: 


ae 1.5 w 
CLUJ; ) = EP] 


a (y27 1. 5)"Y(1. 5 + bos 
[3 bY (aX, - | 
Fema iite foci HRY WAR Peel ee. hema 
(1.5/2 (1.5)? + be? 


é 2 


and expressing P(a) as before by: 


da, 


13] 
P(a) = ate 21 Pn 


27 p, 
this latter expression for CLIs;) may be put in the 
form: 

-Ba* + Ca+ D 


AXE da, 


a 
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This 


and may be approximated as before to yield: 


1.5 "Nbot+(1. 57 


n 

CLG \ SS PICs ya 

i? : | ae Syd. ee 5)° 
le eee eee aL 0" 


e? (1.5)? (1..5)?+be2t+nbp2 (be?+(1. 5)?)(bo2+tnbp2+(1. 5)? 
, (3. 29) 


t 
with differentials dv. i: omitted. Similar expressions exist 


’ 


for CH(S,) and CH(J,.). 


Vi. APPLICATION OF THE THEORY TO SOME ACTUAL 

VIDISSECTOR OUTPUT , 

In this Section we shall apply some computational 
procedures suggested by the foregoing theory to intensity 
values obtained from the vidissector. These intensity values 
will be taken along single lines normal to various lines and 
edges in the visual field. The results of the computation 
will serve to validate the theory presented, and justify 
some assumptions and approximations in its computational 
implementation which will be used by the program described 
in the next chapter. 

In applying the foregoing theory of a regional line 
predicate, it is first necessary to choose a suitable physical 


width for a rectangular region. It appears that, for the 
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vidissector in current use, this width should be between 25 
and 50 units, where one unit is 1/2000 of the full field width. 
This range of values is chosen to conform with the assertion 
that, in the terminology of chapter two, the majority of 
regional evidence for the existence of a line is contained 
within a neighborhood of the line whose radius is on the 
order of a few times the "critical resolution radius" of the 
vidissector. This latter quantity, as defined in Section I 

of chapter two, may be easily obtained by examining 

Figure 3.3. It may be seen that the intensity in the upper 
curve falls to one half of its maximum value at a radius of 
three units from the center of the curve. This value is 
approximately the critical resolution radius as previously 
defined. For present purposes, we took ''a few times'' to 
mean specifically eight times, which yielded a region width 
of 50 units. Later results, reported in the next chapter, 
indicated that this value may have been conservatively large, 
and somewhat better results seem to have been obtained with 
a region width of 25 units. In any case, the value of 50 units 
turned out to be adequate for present purposes. 
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It seems appropriate to begin an empirical investigation 
of the properties of the regional line predicate here developed 
with a consideration of the case where the region to which it 
is applied consists of a single square sub-region. This 
admits of a simplification in the formula for Q*J;) as given 
by (3.13) and (3.29). In particular, the term involving 
(ah. - a) in (3.29) becomes zero, since ar is the 
average of only one term. ‘Thus this term is eliminated from 
the exponent in (3. 29); and likewise similar terms are 
eliminated from corresponding expressions for CE(J2) and 
CH(J}). 

Another simplification of the exponent in the expression 
for cus, given by (3.29), and likewise for corresponding 
expressions for CE(J;) and CH(J;), follows from a 
consideration of the relative magnitudes of the values 
(Ts 5)" and be. The latter is the value of b given by (3. 24) 
times the variance of values of idealized relative amplitude 
of the various lines in scenes in the real world. Since the 
distribution of these values has a mean of zero, it follows 
from elementary properties of the normal curve that about 


1/3 of all lines have idealized relative amplitude a with the 
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property that lal > g.: Since the variance of the sub-region 
amplitudes ay j is small relative to the variance of the values 
of a, the same holds true of these amplitudes, i.e., about 1/3 
of all sub-components of noise-free samples containing lines 
have amplitudes ae with a 2 o. If the intensities 


within a sub-region of a noise-free distorted sample are given 


1 
by (u Lkeoctt? Wy i) and their spread is defined as: 


with 

- t 

cy 2 Yi /m 
then it is easy to see from (3.18) that this spread has the 
value as Nb. where b is as defined in (3.24). It follows 
from what was said previously that about 1/3 of all sub- 
components of noise-free samples have a spread of greater 
than Vbo.. More informally stated: Vbo, is a not uncommonly 
large value of the spread of intensities in sub-components of 
noise-free samples containing lines. In the case of noisy 
samples containing lines, empirical investigations suggest 
that a spread of 30 is not uncommon; and it is easy to see 
from the fact that noise level is relatively much smaller than 
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30, that this value is also not an uncommonly large value of 
spread for noise-free samples containing lines. Consequently: 
bo >> (acer. (3. 30) 
Now by simple algebra the exponent in (3. 29) may be written, 
excluding the third term which is zero, and using the relations 


in (3.23), (3.24) and (3. 25): 


t tt 7} 
1 aah re [pu Vj, ‘1 (0? + np?) 


a (1.5)? (1. 5(1. 5) + bo2+ nbp?) |’ 
It follows from (3. 30) that the coefficient of the second sum 
may be approximated by (1/b(1.5)?), so that the exponent of 
(3.29) is almost exactly equal to: 

1 Lv}, 1) (Buys, i)” 

eae eee : (3. 31) 

2{/ (1.5) b(1. 5) 

A similar sort of simplification of the coefficient of e 
in (3.29) may be made from a consideration of the relative 
magnitudes of nbp. and bo... The value of a. represents the 
variance of a quantity a, idealized relative intensity; and 
0. is, according to assumption 5) of Section IV, the variance 
of small purturbations of the latter. It is reasonable to assume 


2 2 2 
that a0 >> Pp. and that even bo >> nbp*. By this approximation 
n n n 
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it follows that the coefficient in (3.29) is close to: 
P(CL) : (3.32) 
Yb o,( ¥271. 5)" 
If this and the approximation in (3.21) are applied to (3.29), and 


similar approximations are applied to corresponding formulas 


for CE(J;) and CH(J; ); Formula (3.13) reduces to: 


te 1 ras: 

Q*(J,) = 

1 SL(j) 1 SEU) (3. 33) 
P(CL) e 2(1. 5)? 4 P(CE) e (1. 
Yb oa; Vb'o} 

1 SLG) 1 SEG) 1 SH() 
P(CL) , 2 (1.57, P(CE) , 2 (1.5)? , P(CH) ,2 (1.5) 
Yb a, e - Vb'o! € " Yp"'o' e 


Where: 9%, oy: and o1 are respectively the idealized relative 
amplitude variances of elements of CL, CE and CH; 
b =H (ut)’, (3. 34) 
ae Raa SB U") is some paradigm intensity 
pattern for component regions of members of CL; 
b= (uit) (3.35) 
t 
and (u'', ... , U"™") is some paradigm intensity 
profile for component regions of members of CE; 
b" wy’, (3, 36) 
and cu", ... , U'"'™) is some paradigm intensity 
profile for component regions of members of CH; 
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‘ae 4 tt (2 
SL(j) = (QaUty; 1)" /b 

si it. 29a 
SE(j) = (LU v1) Ib 
SH(j) - yt t 2 Jp" 

= Quy 


m y seg ad. ae 
F vi ? are the intensities in J.. 


and ce j 


ae ee 
It is necessary to assign values to the remaining 
constants in (3.33). First we shall assume that lines and 
edges are equally likely, so that P(CL) = P(CE). Also in 
the examples which follow, it will become clear that an edge 
or line occurs in one sample out of 75, so that: 
1/75P(CH) = P(CE) = P(CH). 
Secondly, it follows from empirical investigation of the 
range of spread of homogeneous samples with various 


gradients, that five is a reasonable value for b'o}). It was 


already pointed out that 30 is a reasonable value for ba,, 


(3. 37) 


(3. 38) 


(3. 39) 


and we shall assume this is the case for bie as well. Thus: 


KK 1 
Q (J; ) 


SL(j) SE(j) 
2(1. 5)° 2(1. 5)? 


2(1. 5)? 2(1. 5)? 2(1. 5)? 


e re + 450 @ 
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(3. 40) 


For values greater than .5 this function is nearly 
monotone with: 
he ey ee ee 
Max(Sh()/20.5)-> SEQ)20.5) )= 


SH(j)/2(1. aye ssiano: ) 


Max(SL(j), SE(j)) - SH(j) - 27.5 (3. 41) 
This follows from an observation about the results given later 
in this section, namely that SL(j) and SE(j) are rarely of the 
same order of magnitude. This is not surprising, in view 
of the fact that one expresses the similarity of the obtained 
pattern to a noise free edge, and the other expresses the 
similarity to a noise-free line. For values of QHJ}) 
greater than .5, the pattern must be at least somewhat 
Similar to either an edge or aline. It could not be 
simultaneously similar to both. Consequently the formula 
(3.40) above, which is of the form 


A+B 
A+t+B+C 


has the property that when its value is above .5, A and B 
are considerably different in value, and both positive. 
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It is not difficult to show that in such a case if, for example 
Max(A, B) > 10 Min(A, B), 
then the approximation: 


Max(A, B) Be A+B 
Max(A, B)+C A+Bt+C 


is good to better than 6 percent. The fact that QH55) is 


monotone with Quis) follows from the fact that 


Max(e@ e?) 


Max(e*, e )+ e 
is monotone with Max(a, b)- c, if a, b and c are positive. 

It is desirable to apply the predicate to a variety of 
regions which are homogeneous, contain off-center lines, or 
contain centered lines or edges. We have chosen data from a 
series of areas on the visual field which are 200 units wide 
and contain the images of vertical edges and lines approximately 
centered laterally. The predicate will be applied to 75 
successive overlapping regions 50 units wide, which are 
offset relative to each other by two units and thus exactly 
cover the 200-unit-wide area. The procedure will be applied 
to vertically oriented regions only, since only vertically 
oriented lines exist in the data samples. The fact that 75 
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regions are tested and only one contains a centered line or 
edge justifies the relation P(CH) = 75P(CE) used above. 
Finally, before applying the predicate to real data, 
which will be done by computing the function Qo), it is 
necessary to give values for the sets {U'}, {u't} and {yr} 
used in formulas (3. 34) - (3.39). It will be recalled that 
these are paradigm sets of intensity values at points in some 
fixed geometry within a square sub-region. They correspond 
respectively to a line centered in the region, an edge centered 
in the region, and a homogeneous region. According to the 
fixed geometry which we shall use, the points will be equally 
spaced along a line traversing the square sub-component 
through its center and oriented normal to the expected line. 
These values may be ordered in an obvious manner according 
to their positions along the line mentioned. They are shown 
graphed as a function of this ordering in Figure 3.6. The 
values of b, b' and b" for these sets of intensities are all unity. 
The intensities from the real world will come from single scans 
of 100 points taken normal to a vertical edge or line in the 


real world. The points will be separated laterally by two 
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units, so that the total width of the area scanned will be 200 
units. If it is desired to apply Q**(I5) to some region along 
this scan, out of the 100 intensities Ij; Meas Ti 00° some 


set I. I 


7. ee 5 fall within the region. Then it is 


i+2 

trivial to select intensities from among these to be used as 
1 m : 

the values Vj wort. Ms 1 for use in formulas (3. 37), 


(3. 38) and (3.39). They are simply ly oe I 95 respectively. 
Thus we will be applying (3. 41) to 75 successive 25-tuples from 
the 100 intensities in a particular scan. 

It is important to digress here to point out the 
advantages of the sort of geometry of points within a sub-region 
which is used above. This advantage will be realized in the 
next chapter when it is desired to apply the regional predicate 
exhaustively over the whole field. In particular it will be 
desired to compute the predicate for lines of all orientations 
through any point P on the visual field. Consider those 
situations where the regions are oriented within + 45 degrees 
of the vertical axis of the vidissector, and all pass through 
some point P; and consider only the square sub-region 
containing the point P, which will be termed the p-th 
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sub-region. We shall in chapter IV take intensities relevant 
to regions so oriented, along lines separated by approximately 
the width s of a region, and oriented parallel to the x-axis of 
the vidissector. We may thus assume that there will be a 
unique scan passing within s/2 of the point P; and that the 


intensities I . , I, along its length within a distance s/2 


ye 
of P are exactly the ones from which the ve D will be selected. 
The procedure for selecting the values ve p from among this 
set is parameterized by the angle @ which the region makes 
with the vertical. If the spacing of points in the fixed 
geometry of points is d, then one wishes to choose points from 
among the Ij, --- » LI, Such that their actual spacing is 
approximately d/Cos(Z). If the spacing of the points along the 
scan line is itself d, which will usually be the case in practice, 
then this matching of obtained intensities with those in the 
prototype amounts to ''stretching"' the former by some factor 
between 1 andy2. Thus the various values of, e.g., 

SE(J} p) = 2, v5 a0 which are used in the computation of 
Q*HI;) for various regions containing P, differ only in the 
amount of ''stretching" of one spatial axis relative to the 


other in the matching of intensity values of the scan against 
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those of the set { ut Denoting these various values of 
CE, ‘) by cE p’ @), it turns out empirically that the range 
of these values is rather small; and in particular for |@|< 45°, 
the values are within something like 20 percent of the closer of 
CE; p 10°), CHT, x 30°). One may thus compute these two 
values only, and for any region containing the intensities from 
which they were computed, use one or the other in the 
calculation of Q*(J;) for that region. In fact we shall in 
chapter IV actually compute these two values by stretching the 
Spatial dimension of the paradigm, rather than vice versa. 
This amounts to using two peaks of different widths as paradigm 
profiles for lines, and two edges of different severity of slope 
for paradigms for edges. It should be clear that if the fixed 
geometry of the points were two-dimensional, e.g. ona 
Square grid, no such simple accommodation for the various 
angles might be possible; and doubtless the amount of 
computation involving the intensities in each neighborhood 
would be far more than the calculation of two values. 

To return to the application of Q** to real data, our 
first example, in the upper left of Figure 3.7, comes from 
a horizontal scan across the forward edge of acube. The 
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scan covers approximately one eighth of the apparent width of 

the cube. The cube is a solid metal block which has been 

painted with white paint so as to have a very uniform surface 

and sharp edge. In each of the eight figures, the logarithms of the 
reciprocals of intensity are given as the bottom of the two curves. 
The absolute range of values is 11 units, where 64 units corresponds 
to a factor of two inintensity. It is easy to infer from the physical 
properties mentioned that the slope of the intensity curve on either 
Side of the edge is due to a systematic non-uniformity in illumination, 
rather than to a non-uniformity of reflectance properties. At the 
edge itself there appears to be a sort of ''negative highlight''(a 

local peak). This appears also at the centers of the scans in the 
upper right and upper-middle left examples of Figure 3.7, which 
were taken from the Same cube; and is apparently a property of 
this particular edge. The 100 intensities in the example give rise 
to 75 values of Q** which are graphed above the intensity curve. A 
value of Q@** is derived from 25 successive intensities consisting of 
the one just below it and twelve on either side. Thus a peak in the 
value of Q** should exactly coincide with a peak or discontinuity 

in the intensity profile. The horizontal lines in each of the 

eight figures are placed at distances along the Q** axis which 
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correspond to values of Q* which are indicated on the lines. 

A reasonable threshold value for Q* appears from an 
analysis of this example, and those appearing later in this 
section, to be about 0.1. A cutoff level of approximately this 
magnitude will be used by the program described in the next 
chapter. This cutoff value is chosen on the basis of its giving 
about 70% false positives. There is no simple exact relationship 
between the Q* cutoff and the false positive error rate; however 
it follows from the first theorem of chapter two that if Bis the 
cutoff value for @*, then 1-gis an upper bound of the false 
positive error rate. This may be seen by noting that for the 
threshold procedure, the sum in (2.3) gives the false positive 
error rate. Every element in the sum is some probability 
PCJ;(S)) times a value less than 1-8. Since the values of 
P(J(S)) sum to one, the en in (2.3), i.e., the false positive 
error rate, is less than 1-B. Thus thresholding Q* at 0.1 should 
give a false positive error rate less than 0.9. However, the 
extent to which the false positive error rate is actually less than 
0.9 is an empirical matter. Evidently the various probabilities 
and so forth are such as to give an actual value of 0.7 in this 


case. 
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An obvious question now arises: how subtle may an 
edge or line be relative to the noise level in order to be just 
detectable, e.g., with the threshold of 0.1. An example of 
such an edge is given in Figure 3.7 at the upper right. Itis 
possible to make a general quantitative description of such a 
line or edge by means of a value derived from the cutoff value of 


*%*, Suppose the cutoff value is 0.1; then we have: 


SL(j) 3 


2(1.5)? , 2(1.5 

Max é é ont 
ar. SE(j) SH(j)_ ie 
(1.5 2(1.5 2(1. 57 

Max |[@ ,e +450 E 


- SLUG) SE(j) SH) 
(1.5) 2(1. 5)" 2(1. 5)? 
9, ads ee =~ 450. @ : 
or 
Max(SL(j)/2(1. 5); SE(j)/2(1. 5) + Lun(9) = 
SH(j)/2(1. 5)’+ LN(450. ), 
or: 
Max(SL(j), SE(j)) - SH(j) - 27.5 ~ 2(1.5)Ln(9) = 9.9 
or: 


Max(SL(j), SE(j)) -. SE(j) = 17.6 (3.42) 
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Since SL(j), SE(j) and SH(j) are positive, the minimum value 
of Max(SL(j), SE(j)) subject to (3. 42) occurs when SH(j) = 0. 
This is the case when the net intensity gradient for the sample 
iS zero; a fact which can be deduced from (3. 39) and a 
knowledge of some of the arithmetic involved with simple 
least-squares analysis. In this case for a marginally 
detectable line or edge, SL(j) or SE(j) has the value 17. 6. 
If the intensity profile follows the form, e.g., of the upper 
curve in Figure 3.6; then it is not difficult to see, by the use 
of (3.37), what its actual peak to valley range would be: 
Suppose the 25 intensities in the noisy intensity profile are 
25 


given exactly by aut, ... , aU~. Then SL(j) for these 


values is, according to (3. 37): 


[a y cot? ] a 


» (uty? 
However, the denominator term, which is simply b as defined 
in (3. 34), was chosen to be unity for the set of u's in 
Figure 3.6; so the value of SL(j) turns out to be simply a2, 
This, it was agreed, is 17. 6; so the value of ais 4. 2. 


Thus a marginally detectable intensity profile of this shape 
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is Similar to that at the top of Figure 3.6, but multiplied by a 
factor of 4. 2; and thus has a peak to valley range of about 
three units. In the case that SE(j) is a maximum and the 
phenomenon is edge-like, one may assume the approximate 
shape of the intensity profile is that given by the middle curve 
of Figure 3.6. By asimilar argument, the marginally 
detectable sample of this sort has a range, in this case net 
intensity difference, of about 1. 7 units. The central feature 
of the scan in the upper right of Figure 3. 7 has a value of 
Q* at threshold; and the amplitude of the peak at the center 
may be compared with the former of the two theoretical 
values just referred to. The intensity range for the whole 
scan is 17 units, from which it is easy to see that the peak to 
valley height of the small peak in the center is about four units. 
This compares well with the theoretical value of three units. 
Clearly the amplitudes of marginally detectable lines 
and edges must be considered in relation to the noise level. 
Noise may in general be due both to time noise, random 
fluctuations in intensity measured at a point under constant 
illumination; and to space noise, which consists in random 
fluctuations in intensity as one scans across a surface. A 
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combination of the two may be obtained from an analysis of 
successive differences in intensity of the above samples in 
areas away from the known features. This analysis yields a 
standard error of successive differences of about two in all 
cases. Since the variance of successive differences is twice 
the variance of individual values about a mean, then the 
standard error of the latter should be about 2A/2, or about 
1.6. This is approximately the value of the standard error 
involving time noise only, as reported in (3.5). Consequently, 
the space noise is in this case negligible, which is not 
surprising in view of the deliberate choice of smooth-surfaced 
objects. In any case, the marginally detectable amplitudes 
of 3 units and 1. 7 units for lines and edges respectively 
should be thought of in relation to an underlying noise level 
whose standard error is 1.5. For example, using the 25 
point geometry previously described, one may state that a 
marginally detectable peak has a peak to valley range of 
twice the noise level, and a marginally detectable edge has a 
net intensity difference of about 1. 2 times the noise level. 
Six additional examples are given in Figure 3. 7. 
The left-hand upper-middle scan was taken from the same 
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cube and under the same lighting conditions as the top two in 
the illustration. The other five were obtained from a scan 
across the vertical forward edge of a second cube under 
various lighting conditions. The lower-middle pair, marked 
NEDGE2A and NEDGESA, were taken under identical lighting 
conditions, but at different heights along the vertical edge. 
This cube is uniformly painted, but the edges are slightly 
rounded, accounting for the pronounced highlight (dip in 
intensity, since the values are actually Log(1/intensity)) 
apparent in some of the profiles, particularly the one marked 
NEDGE2A. 

There is a certain weakness in the use of so great a 
region width as 50 units, namely that the intensity profiles 
across a region may have non-negligible gradients toward 
the edges of the region, appearing, e.g., like the profiles 
in Figure 3.8. For this reason, it is useful to generalize 
the thresholding procedure so far described by computing 
values SL, (j), ase SL,(), SE,(j), ... , SE,,(j) for a set 
of n peaked profiles and m edge profiles, and using a 
maximum of all these values in place of the maximum over 
two elements as in 3.41. Denoting this generalized version 
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of Q** by Q'**, we may express it in two slightly different 


forms: 
QUT) = 
Max (SL,(j), SE,(j)) - SH(j) - 27.5 = 
i,k 
Max (SL;(j) - SEG), SE,(j) - SEC) - 27.5. (3. 43) 
1, 


This generalized formula will be used in the program described 
in the next chapter. In the remainder of this chapter we will 
discuss its application to the previously analyzed data of 
Figure 3. 7. 

For present purposes, we shall consider only a possible 
multiplicity of paradigm peak profiles. Four additional 
paradigm profiles besides the one in Figure 3.6 are illustrated 
in Figure 3.8, and denoted by WPEAK, CPEAK, BPEAK, and 
APEAK respectively. It is instructive to compute individually 
the values whose maximum gives the value of Q'** according 
to the second expression in (3. 43). In particular we shall 
consider six values in the following order: 

SL, (j)-SH(j), corresponding to the paradigm WPEAK; 

SL,(j)-SH(j), corresponding to the paradigm CPEAK; 
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SL,(j)-SH(j), corresponding to the paradigm BPEAK; 
SL,(j)-SH(j), corresponding to the paradigm APEAK; 
SE(j)-SH(j), corresponding to the edge paradigm of 

Figure 3. 6; 

SL, (j)-SH(j), corresponding to the peak paradigm of 

Figure 3. 6. 

In Figures 3.9 and 3.10, we have these six ''components"' of 
Q'** individually graphed above the corresponding intensity 
profile, in the above order top to bottom, in a manner similar 
to that used in Figure 3. 7. 

One may make a series of observations concerning the 
various curves in Figures 3.9 and 3.10. Comparing the 
lowermost two component curves in the upper left of Figure 3. 9, 
it is easy to see that the pair of peaks in the upper left 
illustration of 3. 7 were due to the peak-detector profile 
entirely. This follows from the fact that the peaks appear 
only in the lower of the pair of curves from 3. 9 just mentioned. 
Regarding the same profile, one may see that the major peak 
from among the component curves is the uppermost. This 
corresponds to the profile WPEAK, which is a wide peak. 
Evidently the feature represented by the maximum of this 
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curve is the inverted wide peak to the left of the center of the 
intensity scan. Another observation concerns the lower right 
example in 3.9. It is reassuring to note that although the 
discontinuity present in the intensity profile is quite large, 

it was ''noticed" by none of the peak-detector profiles. This 
is evidenced by the fact that in the top four component curves, 
and in the bottom one, the curves are not above threshold in 
the vicinity of the extreme intensity discontinuity. On the 
other hand, the next-to-bottom curve, corresponding to an 
edge, has an extreme peak at this location. Lastly, one may 
make the general observation that the maximum of all six 
component values of Q'** may be thresholded at a considerably 
higher level than Q**, for example at a value corresponding 
to aQ'* value of .99, and still detect all edges, but ata 
considerably better signal-noise ratio. Observation of the 
figures shows that at this level a few false positives remain, 
e.g., the peak to the right of the large discontinuity in the 
lower right example in Figure 3.9. An examination of the 
intensity curve itself would appear to justify the contention 
that there really was "something there" after all, perhaps 

a spot on the cube, or an insensitive spot on the vidissector 
photocathode. 
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CHAPTER 4 


A COMPUTER PROGRAM FOR FINDING LINES 


I INTRODUCTION 


Figure 4.1 


Figure 4.1 illustrates the first stages in the analysis 
of scenes by the program to be described in this chapter. On 
the left is a photograph of a set of objects placed before a 
random access optical input device connected to a digital 
computer. Programs in the computer have access to 
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intensities in the field of view at arbitrary locations ona 

20000, by 20000, grid. Intensities are obtained by the program 
along 50 vertical and 50 horizontal scans of 500 intensities each. 
The intensities are converted into ''feature points", as 
illustrated in the center illustration of 4.1. Feature points are 
locations where the scans are adjudged, by spatially local 
processing, to intersect with an edge of an object in the real 
world. Often noise in the real world or within the vidissector 
will produce an intensity configuration along part of a scan 
which is edge-like and gives rise to a ''false positive" feature 
point. The right-hand portion of Figure 4.1 illustrates the 
output of a program which extracts lines from an array of 
feature points and which is designed to ignore these 

extraneous ''false positive'’ points by detecting chains of 
lined-up feature points. A final stage of the analysis, not 
illustrated in Figure 4.1, is the proposing and verification 

of lines not located by the above procedure. Lines are proposed 
on the basis of lines already located, in places where figures 


appear to be incomplete. 


II. A DESCRIPTION OF SCENES ANALYZED 


In this chapter we will describe an object recognition 
program based on the theory so far developed. This section 
will be devoted to a discussion of the restrictions imposed 
by the theory on the scenes analyzed, and on the optical input 
device used. Various scenes conforming to the restrictions 
will be illustrated. 

The theory developed in the preceeding chapters is 
applicable to certain types of scenes, and to optical input 
devices satisfying certain conditions. Stated informally, the 
restrictions on the scenes are: 

1.1) The scene consists entirely of relatively homogeneous 
regions bordered by edge lines. 

1.2) The edge lines are all straight. 

1. 3) The objects of the scene may be recognized entirely 
from the edge lines. 

The first of these restrictions is fundamental. It is 
virtually equivalent to the assumption that an intensity profile 
across a scene consists of a series of smooth curves bounded 
by discontinuities as in Figure 4.2. 
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INTENSITY | "~~ — 


HYPOTHETICAL INTENSITY PROFILE ACROSS A 
SCENE IN THE ABSENCE OF BLURRING AND NOISE 


Figure 4. 2 

The second of these assumptions is essential if one is 
applying an edge test only to narrow rectangular regions, as 
is the case in the program under consideration. Of course 
one could develop a theory, analogous to the foregoing, for 
amalgamating local feature point information over curved bands, 
and thereby detect edge lines of arbitrary shape. However we 
have not done this; it almost certainly would require computation 
of an order of magnitude above that used in the current program. 

The third restriction is a matter of convenience. 
Actually the program may be presented with scenes not 
conforming to this restriction without any impairment of its 
behavior. For example, it might be presented with an edge-on 
view of a cylinder, and successfully recognize the outline. 
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The roundness, though detectable by the vidissector from the 
intensity gradient across the surface, would not be detected by 
the present program. 


These restrictions are summarized in Figure 4.3. 


NOT APPROPRIATE: 
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ATTENTION WILL BE LIMITED TOA 
CERTAIN CLASS OF SCENES 


Figure 4.3 
The theory requires certain restrictions on the 
instrument which obtains the intensities from the real world. 
These are, in Summary: 
2.1) The extent to which the input device blurs the image 
received must be uniform over the whole field. 
2.2) The extent to which the intensities are modulated by 
noise must be uniform over the field, and independent 
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of relative intensity. 

Stated more Specifically, the first assumption says 
that if I(x, y) represents the intensity of the scene over the 
field of view, D; then the intensity received by the optical 


input device in the absence of noise is: 
I*(x, y) = fix-v, y-v)f(u, v) dudv 
D 


for some blurring function f(x, y). 


These assumptions are summarized in Figure 4. 4: 
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SUBJECTED TO BLURRING AND NOISE 


Figure 4.4 
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Some scenes analyzed by the program are shown in 
Figure 4. 5. 

It is easy to see that criteria 1.2 and 1.3 are 
satisfied by scenes of this sort. However, it is not readily 
apparent that criterion 1.1 is satisfied. This follows from 
the fact that real lines such as highlights and cracks between 
cubes, in the absence of blurring and noise are never of zero 
width as stipulated in the definition of aline. However it is 
easy to see that they may be of nonzero width so long as they 
are sufficiently narrow to be indistinguishable from zero-width 
lines under the blurring function. This blurring function was 
discussed in Section II of chapter three. From its properties 
it is easy to see that blurred versions of lines of less than 
about five units width, where one unit is 1/2000 of the 
vidissector field width, are virtually indistinguishable from 
each other and from blurred versions of lines of negligible 
width. Criterion 1.1 has thus been satisfied for these figures 
by keeping the apparent widths of the edge lines below the 
aforementioned five units. 

Criterion 2.2 has been met by the optical input device 
used, an Information International vidissector, by the nature 
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EXAMPLES OF SCENES ANALYZED 


Figure 4.5 
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of its design. However certain care is required to satisfy 2.1, 
and it can not be completely satisfied by the instrument used. 
The blurring inherent in the vidissector is a function 
of optical focus, of internal electronic focus, and of the size 
of the internal aperture used to collect intensity data from a 
point in the visual field. The latter two matters have been 
discussed at length by Horn (Horn 1969), and in the literature on 
the instrument itself. Suffice it to say here that there is a 
more or less uniform degree of blurring inherent as a result 
of these two factors, though a certain amount of non-uniformity 
is detectable by careful measurements. It is fortunate that the 
order of magnitude of these effects is somewhat greater than the 
optical blurring of the system when it is focused to minimize 
this blurring. By keeping the objects in the field of view ina 
sufficiently flat plane, it is possible to have the non-optical 
blurring effects dominate the optical effects even for portions 
of the field which are relatively out of focus. This, to a first 
order of approximation, insures a uniformity of over-all 
blurring, despite the fact that the purely optical blurring is 


1 


quite non-uniform. The criteria for ''a sufficiently flat plane" 
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for the image is given by laws of optics. From the well known 
relation that: 

1/F =1/a+1/b 
Where: 

a = object distance 

b = image distance 

F = focal length of the lens, 
it follows that: 

da/db = - a? |b". 
Hence for a given displacement Aa of the object from perfect 
focus, the image displaces by an amount hale? yar): This 
quantity, divided by the focal ratio f, gives the amount an image 
is blurred by displacing the object a distance Aa from perfect 
focus. Thus the allowable scene depth for a particular 
maximum allowable amount of blurring depends on the square 
of the scene-distance image-distance ratio; and inversely on 
the f-ratio. If a particular object or arrangement of objects 
increases in absolute size, and its distance from the lens 
increases proportionately (and if the object distance is large 
relative to the lens-image distance), its image remains 
approximately constant in size, but the depth of the object 
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remaining in good focus increases linearly relative to the size 
of the object. It is thus possible, for a given arrangement of 
objects to keep the blurring over the whole field down below 
any pre-assigned limit by making the f-ratio sufficiently large, 
and by making the objects sufficiently large. In practice, due 
to the low sensitivity of the vidissector it was often impossible 
to insure adequate depth of field by the use of a suitably small 
aperture alone. In particular, in the case of the objects 
illustrated in Figure 4.5, it was necessary to make an effort 
to make the objects as large as possible to satisfy criterion 2.1. 
II. THE DETERMINATION OF FEATURE POINTS FROM 

INTENSITY INFORMATION 

In this section we will be concerned with a discussion of 
a procedure for extracting information from a set of intensities 
taken over a visual field, as a first step in the determination 
of the locations of lines in the field. The relevant theory, 
developed in the previous chapter, will be reviewed; and an 
algorithm for reducing intensity information to "feature points" 
will be discussed. 

In applying the line predicate to various regions on 
the visual field, we shall obtain intensities along scan lines 
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perpendicular to the axes of the vidissector. In particular we 
shall use intensities obtained from scans parallel to the x-axis 
in applying the line predicate to long rectangular regions 
oriented within + 45° of the y-axis. Similarly, whenever a 

line predicate is applied to a region within + 45° of the x-axis, 
intensities from scans parallel to the y-axis will be used. We 
may thus henceforth without loss of generality assume that we 
are dealing with regions oriented within + 45° of the y-axis, and 
with intensities obtained from scans taken parallel to the x-axis. 
A scan shall consist of 500 intensities taken at intervals of 

2 units along a line, where, as in the previous chapter, one 
unit is 1/2000 of the full field width. The line is centered 
laterally in the field. Fifty scans will be taken at intervals of 
20 units, with the 25-th scan half way up the field of view. The 
set of scans will thus encompass a square area exactly centered 
in the field of view, whose edge length is 1000 units, or one 
half the edge length of the entire field of view. The reason 

for not using the entire field is simply that there are various 
physical problems at the edges, one of which involves an 
aperture which prevents obtaining intensity values at the corners. 


127 


The central square used is approximately the largest useful 
square region in the field of view. For purposes of visualizing 
the following explanations, it may be assumed that an entire 
set of 2500 intensities is obtained at once, prior to any further 
processing. This is not entirely true, but may as well be 
considered to be the case. 

We will want to be able to apply the line predicate to 
any of approximately fifteen million rectangular regions in the 
field of view. This figure is obtained from the fact that we 
are considering regions extending between pairs of scans, and 
the latter number 50.49/2 = 1250; we are considering regions 
at any of fifty orientations; and we are considering regions 
with any of 500 different intercepts. In fact we shall apply a 
modified version of the predicate to all of a subset of these, 
namely those which traverse the field entirely, which number 
50-500 = 25,000. This will be discussed in SectionIV. Also 
the predicate will be applied to shorter subregions of those 
among the latter which appear to contain a line somewhere 
along their lengths. Further the predicate will be applied to 
regions proposed on the basis of lines already found. This will 
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be discussed in Sections V, VI and VII. 
The application of Q* to a region on the field of view 
consists in computing the value given by (3. 13): 
Q*HI;) = 
CLIJ;) + CE(I2) 


3 


CL(J;) G CE(J}') 7 CH(J}) 


where CL(J}) is given by (3.29), and CE(J;) and CH(J;) are 
given by similar formulas. A region whose maximum minus 
minimum y-distance is s units intersects with lls/20llscan 
lines, where ||ill means the closest integer toi. Instead of 
assuming that a region is of a particular fixed width measured 
perpendicularly to its length, we shall assume it has a fixed 
x-direction width of 50 units. Consequently the intensities 
along a particular scan which lie within a rectangular region 
will be exactly 25 in number, since the separation of intensity 
points along a scan is two units. We shall also assume that 
the length of a sub-region, instead of being a fixed quantity 

as measured along the axis of a region, has a fixed y-direction 
length of 20 units. Also sub-regions will be taken to be 
vertically centered on a scan line. This arrangement is 
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diagrammed in Figure 4. 6. 


SUB-REGIONS 
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Figure 4.6 

We may thus identify a segment of a scan enclosed within a 
region with a particular sub-region. The advantages of this 
sort of geometry were discussed at the end of the previous 
chapter. We shall use a specific term for the segment of a scan 
which we are associating with a particular sub-region, namely 
a sub-region scan segment. In Summary, a region of length 
a consists of lla/20I| = m sub-regions; and the value of Q* is 
computed from the m sub-region scan segments by calculating 
values of CL, CE, and CH according to formulas of the form 
given by (3. 29). 

Evidently the intensities of a particular sub-region 
scan segment enter into the computation of Q* for several 
thousand regions enclosing the segment. Examining (3. 29) 
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it is apparent that, e.g., in computing CL, it is necessary to 


compute a value R. , from the intensities in this segment, and 


ji, 


also a value a i? and that CL is determined entirely from these 
values. It is not hard to see that these values are the same for 
a particular segment, when the enclosing regions are all at the 
Same inclination. Further, it was pointed out in some detail 

in the last chapter that these values for a particular segment 

do not vary a great deal with orientation of the enclosing 
rectangular region assuming, as we are, that the regions 


remain within + 45° of a normal to the scan lines. Thus, as 


before, only values of R. 


we etc. , for two region orientations, 
J» 


10° and 30° , need be computed. This is done, as in the previous 
chapter, by using two paradigm profiles of different widths. 
Finally, we may recall that at the end of the previous chapter 
the procedure of computing a single value of R, R', etc., for 

a particular segment and orientation was generalized to the 
computation of several, corresponding to different possibilities 
of the shape of the intensity profiles toward the edge of the 
region. Consequently, for a given sub-region scan segment, 


we will compute exactly the values R. R 


i, Ky 1? ene j,k, 6’ 


1 


ax Pe ee a: » R! Oo » R » ale BD AS Ee 
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a ze and also R" - and a The six R's and a*'s are 
J, ’ Js J; 


computed as in (3. 25) and (3. 23) from six different peaked 
2 
profiles (we aS nin u ved (Up. nite “4 us") of various widths 


and with various combinations of gradients towards the ends. 
Similarly the R''s and a'*'s are computed from five different 
cliff-like profiles. Clearly, no such multiplicity is necessary 
in the case of R'' and a''*, which corresponds to a homogeneous 
region. These paradigm profiles, 25-tuples of values (UL. 


25 
Seige ke 


1 ), ete., from which the R's, a*'s, R''s, a'*'s, and 


R" and a''* are computed, are illustrated in Figure 4. 7. 


Since values of R. » ak Te ae ee oe alk P 
j,k, 1 ek, j,k, oi j,k, 5 
Ri ye and a will be required for every sub-region scan 


segment, they are all computed. Since there are 475° 50 
sub-region scan segments, and 24 values computed per segment, 
this is a matter of 570, 000 intermediate values, an order of 
magnitude more than the 25,000 intensity values from which 

the values were computed. However, the derived set of 

values admits of considerable compactification. First, for 

a particular sub-region scan segment, it is necessary to retain 
only the minimum of all the R's and R''s, together with the 
corresponding value of a* or a'*. Secondly, consider the 
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values of R'' minus this minimal R or R' for the 475 successive 
sub-region scan segments along a particular scan. Then there 
exists a threshold for these differences such that most values 
lie below the threshold and indicate with virtual certainty that 
a line does not traverse a region containing that segment. 

Such values may be omitted, thereby indicating that they fall 
within this category. Also, only the local maxima of the values 
above threshold need be considered. Consequently an entire 
eeiieeuen of 475 sets of values may be reduced to somewhere 
around ten sets. These latter sets need only consist of 4 
values: the value of R'' minus the minimal R or R' which is 
locally maximal; the value of a* or a'* corresponding to the 
minimal value of R or R'; an indicator as to which R or R' 

was minimal; and the x co-ordinate of the point in question. 
Such a four-tuple, together with the y co-ordinate of the scan 
from which it was derived, will be termed a feature point. 

We will not justify in detail here why the 570, 000 values can 
be reduced to approximately 500 feature points. However we 
shall point out that the subsequent use made of derived 
information in the computation of Q* and similar values uses 
no more than the reduced feature point data; that many of the 
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values can be considered to be in a single null category; and 
that the remaining values have sufficient regularity to be 
specifiable in terms of their local maxima only. 

The program (GETFP X 8S) obtains intensities for a 
single scan and performs the calculations sketched above. 
The second argument is 1 or 0 depending on whether a scan is 
to be made vertically or horizontally respectively. The value 
of X is the appropriate ordinate or abcissa ona scale from 0 
to 500. Five hundred intensities are taken across the field 
along the line defined by the arguments and are processed 
according to the algorithm sketched above. The output is a 
series of 4-tuples as described above. A schematic diagram 
of this process appears in Figure 4. 8. 

In the line finding program which has been developed, 
the sub-program GETFP is called 50 times for vertical scans 
and 50 times for horizontal scans. This procedure is illustrated 
in Figure 4.9. In this figure, the short lines indicate the 
locations of maxima previously described. Their lengths are 
proportional to the corresponding values of R'' - Min(R, R') 
for small values of this difference. For values of this difference 
above a certain threshold, a constant length of line is used 
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Figure 4. 9 
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corresponding to this threshold value. These points will be 
termed feature points. Rasters of feature points corresponding 
to the illustrations in Figure 4.5 are given in Figure 4.10. 
IV. A GLOBAL PROCEDURE FOR EXTRACTING LINES 
FROM ARRAYS OF FEATURE POINTS 
If one were to examine the feature point rasters of 
Figure 4.10, one would be able to pick out almost all the lines 
in the figures they represent. The perception of some of the 
lines may depend on complex global perception processes, 
for example, the use of obvious lines to provide clues as to 
the existence of less obvious ones. A computer approximation 
to this process will be the subject of subsequent sections. On 
the other hand, some of the lines are obvious without either 
reference to other parts of the figure or a priori knowledge 
of what constitutes a plausible figure. This is because the feature 
points are both strong (represented in 4.10 by long lines), and 
they line up very exactly. This section describes a program, 
LINES, which takes advantage of the obviousness of certain 
lines to find them in an array of feature points, by a rather 
simple and not overly lengthy procedure. 
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FEATURE POINTS EXTRACTED FROM THE SCENES IN FIG. 4.5 


Figure 4.10 
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The problem of locating lines by accumulating locally 
obtained information has been approached by line following 
procedures. (Roberts) This often runs into the problem that 
the line follower gets lost, since, in effect, it is making a 
series of decisions (whether or not to continue the line) on 
the basis of local considerations. This problem becomes 
greater as the number of lines in the scene increases. Line 
followers are also subject to the problem of never having 
"attached'' themselves to certain lines in the first place. It 
is thus desirable to use a more exhaustive and global procedure, 
for example, covering the feature point raster with a very large 
number of narrow rectangles, and applying a thresholded 
predicate to the feature points within each rectangle. The 
apparent tremendous cost of such a procedure can be greatly 
reduced by observing that if a certain narrow rectangle contains 
evidence of a line, then so does a rectangle with the same 
orientation but extending across the entire field. The converse 
of this statement is not true, since a rectangle across the 
whole field may contain a large amount of line evidence in the 
form of scattered noise and spurious values due to feature points 
along lines intersecting the rectangular region at some angle. 
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However, the testing of narrow rectangles across the whole field 
does provide a sort of screening procedure. Any relatively 
strong lines will fall within at least one of these bands, and thus 
be detected. On the other hand, it is necessary to further analyze 
the contents of one of these bands which has a large amount of line 
evidence in it in order to determine both if there really is a 

line within the band; and if so, where it is. 

The procedure for assessing the total amount of evidence 
for the existence of a line within a band is based on the idea of 
projecting a two dimensional array into a one dimensional line. 
For example, if an array of points (x;, y;) is given, and it is 
desired to determine the locations (x-intercepts) of vertical 
bands containing a large number of points, then it is merely 
necessary to histogram the x-co-ordinates of all the points. 

The resulting histogram has large values for x-intercept 

values of bands containing a large number of points. By this 
procedure, all vertical bands are investigated in parallel. 

The procedure is simply modified for arbitrary orientations, 

by histograming numbers of points versus a quantity 

x, + ay; where a depends on the orientation under consideration. 
This corresponds to projecting the raster of feature points onto 
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the x-axis at some angle. 

The program (PROJ X SL) carries out this projection 
procedure on an array of feature points X, projecting at an 
angle SL. The main program calling PROJ uses 100 values of 
SL. For 50 values, the feature points obtained from horizontal 
scans (parallel to the x-axis) are used. The values of SL in this 
case correspond to orientations between + 45 degrees with the 
vertical (the y-axis). The remaining 50 projections use the 
feature points from the vertical scans, and involve angles 
between + 45 degrees with the horizontal. The values histogramed 
are not Simply numbers of points, but sums of integer weights 
attached to the various feature points. These integers have 
values from 1 to 4, and depend monotonically on the value of 
R'" - Min(R, R'), the first feature point value, for a particular 
point. Thus, for a particular band, it is not simply the quantity 
of feature points within it that is considered, but something 
approximately equal to the sum of the values of R'' - Min(R, R') 
for points within it. This gives extra weight to distinct lines, 
and may be shown to approximate Q* in some sense. The value 
returned by (PROJ X SL) is the set of x-intercepts of centerlines 
of bands of slope SL which have a large amount of line evidence 
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in them. The algorithm employed by PROJ is diagrammed in 
Figure 4.11. 

The set of outputs of (PROJ X SL) must be analyzed to 
extract the actual lines in the scene. One problem which must 
be taken into account arises from the fact that a particular set 
of feature points may give rise to an intercept value in the 
output of several applications of PROJ, for successive values 
of SL. This is a result of the fact that a given set of feature 
points which fall in a line along the visual field are contained 
within several rectangles of almost identical slopes. Another 
problem is that the output of PROJ contains a large number of 
false-positive values due to spurious effects described 
previously. 

The program LINES, which calls PROJ for various 
values of the arguments, takes these factors into account by a 
complex process of: 

1) retrieving the feature points in sets from areas 


suggested by the output of PROJ, 


2) eliminating sets which do not actually constitute lines, 
3) eliminating spurious points from sets which do represent 
lines, 
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Figure 4.11 
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4) sorting by the apparent strength of lines, 

5) eliminating redundant sets of points by considering the 
set in the order imposed by step 4 and eliminating seis 
which are highly redundant with (more distinct) sets 
which had previously been considered, 

6) fitting lines to the resulting sets of points. 

The output of (LINES X) is a set of lines retrieved from the set 

of feature points X. LINES is applied once to the feature points 

from the vertical scan; and once to the feature points from the 


horizontal scan. The result of applying LINES to the sets of 


feature points illustrated in Figure 4.10 is shown in Figure 4. 12. 


Ve LINKING LINES TO FORM PARTIAL FIGURES 


Having located a certain number of lines in a scene by 
the methods of the previous section, a logical next step is to 
join them together and form figures. This may be viewed as a 
terminal step, provided all lines have been located by the 
projection procedure. This would not, in general, be the case; 
and the joining procedure described in this section is actually 


a first step in the location of the remaining lines of the figure. 
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We shall assume that the output of LINES, when applied 
to a set of feature points, does not include all the lines of 
interest in the original scene. This is, in fact, the case for 3 
out of 4 scenes in Figure 4.12. It is a reasonable hypothesis 
that the lines found, together with some assumptions about the 
nature of the scenes examined, provide some clues as to the 
locations of the remaining lines. It appears not to be the lines 
individually which suggest the locations of other lines, but 
rather collections of lines which form a figure which is 
incomplete relative to a model of how the actual figures appear. 


An example is shown in Figure 4.13. 


THE SOLID LINES HAVE BEEN IDENTIFIED, 
THE DOTTED LINE IS SUGGESTED BY THEM. 


Figure 4.13 
It would be desirable to join the lines into groups which would 
suggest the possible location of other lines. A line verifier 
could then be applied to the locations suggested. This 
procedure requires that all locations of missing lines ultimately 
be proposed. Otherwise, some lines would never be found by 
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either procedure. It is thus important that the whole mechanism 

for line proposing be liberal in proposing lines. Any excess lines 

proposed will almost certainly be rejected by a suitably powerful 
verifier. 

Prior to the application of the linking procedure, a 
merging procedure is applied to the lines. This is necessary 

to eliminate redundant lines from two sources: 

1) Lines which are at approximately at a 45-degree angle 
with the vertical, and appear in the output of LINES 
applied to both the vertical and the horizontal scans. 

2) Single lines which appear, for one reason or another, 
as a pair of contiguous lines in the output of LINES. 


The criterion for merging lines is given in Figure 4.14. 
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Figure 4.14 
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If two lines are to be merged by this criterion, the pair is 


replaced by a single line which best fits the endpoints of the 


two original lines. 


The joining procedure consists in applying directed 


links from one line to another. There are four possible types: 


1) 


2) 


Regular links, denoted by the symbol T. If any part of 
line B lies within a neighborhood of radius 10 units 
around A, then A is given a directed T-link to B. 

This is illustrated in Figure 4.15. 

Extension links, denoted by the symbol E. If any part 
of line B lies in a particular region near the end of A, 
or if the end of line B lies within another more distant 
region extending away from the end of line A, then an 
E-link from A to B is established. These regions are 
illustrated in Figure 4.15. The motivation for this type 
of link is that possibly a segment of A extending to B, or 
a longer segment extending to the end of B, has been 
omitted. The reason for postulating a rather long 
extension to the end of another line is that a line's 

end, lining up with a given line, provides strong 
evidence that the original line may actually extend to 
that end. 
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not linked 


we regular link 


extension link 


passive link 


A LINK IS ESTABLISHED FROM LINE A 

TO LINE B ON THE BASIS OF THE INTERSECTION 

OF B WITH CERTAIN REGIONS AROUND A 

Figure 4.15 

3) Passive regular links, denoted by PT. If line B links 
to line A by a ‘I-link, but A does not link to B by either 
type of link, then a PT-link is established from A to B. 

4) Passive extension link, denoted by PE. This is similar 
to a PT-link, but is applied if B links to A by an E-link. 
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An illustration of the links applied to one of the sets of 
lines from Figure 4.12 is given in Figure 4.16. It Should be 
emphasized that, in accordance with the necessary liberal 
policy underlying the proposing of lines, the lines are over-linked. 
These links are not intended to be those applied when all the lines 


have been located. 


VI. HEURISTIC LINE PROPOSING 


This section describes the program (PROPOS X), which 
proposes additional lines from a possibly incomplete set X of 
lines from a scene. Each line is considered in turn, and 
additional lines suggested by it, and by lines linked to it, are 
proposed. A verification program, to be explained in the last 
section, is applied to each proposed line. If the verifier claims 
that the line actually exists, it is added to the list of lines, 
together with links between it and other lines; and it may be 
used subsequently in proposing further lines. When no more 
lines can be proposed from the (possibly augmented) set of lines, 
PROPOS returns the entire set. 

The proposing procedure is based on the premise that if 
aline is missing from a set 5S, then at least one line having a 


151 


(link) 


LINKS BETWEEN MEMBERS OF THE OUTPUT OF LINES 


Figure 4.16 
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common end-point with the missing line is in the set X, or can 
be successfully proposed from X. This is illustrated in 


Figure 4.17. It is further assumed that vertices have no more 


(missing)7 


IT IS ASSUMED THAT IF A LINE OF THE 
ORIGINAL FIGURE WAS NOT AMONG THOSE 
OUTPUT BY LINES, THEN AT LEAST ONE 
LINE CONNECTING TO IT WAS. 
Figure 4.17 

than three or possibly four lines radiating from them. It 
follows that lines should be proposed from the ends of lines 
already found, provided that there are not already many lines 
incident with a particular end. 

It is in general not entirely clear where a line generated 
by LINES actually terminates. This is partially due to the 
rough ''mesh" of the scanning grid, and partly due to the fact 


that the feature point generator gets confused in the neighborhood 


of avertex. In order to propose lines from the ends of known 
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lines, it is necessary to make a reasonable estimate as to 

exactly where the ends lie. The first step in proposing lines 

from a given line is to dispose of the ends in the following 
manner: 

1) If two lines intersect within ten units of the end of a given 
line, the point on the line (possibly extended) closest to 
the intersection is taken as the end or ''terminus". 

2) If a terminus is not determinable by criterion 1, buta 
line intersects the given line within 25 units of the end, 
this intersection is taken as the terminus. 

3) If no terminus can be found by 1 or 2, that end of the 
line is said to be of "indeterminate terminus. "' 

The determination of termini is diagrammed in Figure 4. 18. 
Another assumption made by PROPOS, which is 

independent of the first, is that the opposite sides of faces of 

the figures are approximately parallel. If it is desired to 
propose a line extending, e.g., downward from the left 
terminus of a horizontal line, the direction of the proposed 
line is then the same as that of another ''secondary" line 


extending downward from the given line. In particular, this 
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THE "TERMINI" OF THE HORIZONTAL 
LINES ARE AS INDICATED, - 

LINES PROPOSED FROM THE HORIZONTAL 
LINES EXTEND FROM THE TERMINI. 


Figure 4.18 
"secondary" line should be the first line ''to the right" of the 
left terminus, as that is the obvious candidate for an opposite 


side to the proposed one. This determination of these secondary 


lines, termed "directors" is diagrammed in Figure 4. 19. 
Having determined the termini and directors for a 

particular line, the procedure for proposing lines from a 

particular end is governed by four cases: 

1) The end of the line is of undetermined terminus; 
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UPPER DIRECTOR 
FOR LEFT END OF A 


LINE "A''A 
LOWER DIRECTOR 
FOR LEFT END OF 
LINE A 


NOT DIRECTORS 
FOR LEFT END 
OF LINE A 


DIRECTORS ARE LINES WHOSE DIRECTION 
SUGGESTS THE DIRECTION OF A MISSING LINE, 


Figure 4.19 
2) Only one line intersects with the given line at the 
selected end; 
3) Two lines lying on the same side of the given line 


intersect the selected end; or 
4) Something else occurs at the selected end. 

In the first case lines are proposed from either side of 
the original line for which a director has been found. Three 
parallel lines are proposed, 10 units apart, parallel to the 
director, and with the end of the middle line coinciding with 
the indicated end of the given line. The other ends of the 


proposed lines are determined in this case, and in the cases 
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which follow, by making the proposed line no longer than the 
director, and truncating it so that it does not intersect any other 
lines in the set. In case 2), lines are proposed on the opposite 
side from the intersecting line, provided a director exists on 
that side. Three lines are proposed ten degrees apart, all 
extending from the terminus. Their lengths are determined 

as above, and the middle line is again parallel to the director. 
If no lines are found on the side away from the intersecting line, 
then lines are similarly proposed on the same side as the 
intersecting line, subject to the additional provision that the 
proposed lines all lie between the intersecting line and the 
director. If this were not the case, it would be impossible that 
the proposed line and the director be opposite sides of a face. 
In case 3), lines are proposed similarly to the first set 
described for case 2). In case 4), no lines are proposed. 

These cases are diagrammed in Figure 4. 20. 

In summary, PROPOS goes down the list of linked lines 
proposing in the manner described above, adding lines which it 
finds to the end of the list. This procedure is applied to the 
resulting list, and so on until no lines on the current list result 
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1) LOWER TERMINUS OF "A" IS UNDEFINED, 
PARALLEL LINES PROPOSED: DIRECTORS FOR 


LOWER END OF "A" 


ieee ——— PROPOSED 
2) ONE LINE INTERSECTS LOWER END OF "A", 
RADIAL LINES PROPOSED: 
o TERMINUS 
LINE ''A"' ———— PROPOSED 
LINES 


. LINE "B" 


‘SPROPOSED LINE TRUNCATED BECAUSE OF "B"' 


3) TWO LINES AT LOWER END OF "A" ON ONE SIDE, 
RADIAL LINES PROPOSED ON OTHER SIDE: 


o TERMINUS 


—'-—— PROPOSED 
LINES 


4) LINES ON EITHER SIDE OF LOWER END OF "A", 
NO LINES PROPOSED: 


LINE "A" 


PROPOSING OF LINES FROM AN END OF A GIVEN ONE 
IS GOVERNED BY FOUR CASES, 


Figure 4.20 
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in the proposing of any further lines. At this point, the final 
set of lines is returned by the program. The process of 


proposing lines is diagrammed in Figure 4. 21. 
VII. A LINE VERIFIER 


Line verification consists in applying the function 
Q¥(5;) as given by (3.13) and (3.29), or some variant of it, to 
the intensities se or some values derived from them, lying 
in a region suggested by the line proposer. The region is 
assumed to be divided into n square or nearly square sub- 
regions, and Q* or its variant is computed from the values 


R . , etc., according to Formulas 


eae Roa! ae Ses 
(3.13) and (3.29), or formulas derived from them. In most of 
the remainder of this section we shall be concerned with a 
generalization of Q*(J}) which has been suggested by the 
discussion at the end of chapter three. This generalization 
consists in assuming that the values of Ry , and ae , are 
given by: 


es) 
mM 
= 
= 
a 


R ), (4. 1) 


(4. 2) 


ou 
{t 
= 
B 
a 
a 


OUTPUT OF LINES APPLIED TWO PROPOSING PASSES 


TO FEATURE POINTS IN 4.9 


THREE ADDITIONAL FINAL SET 
PROPOSING PASSES OF LINES 


DIAGRAM OF THE PROCESS OF LINE PROPOSING. 
SMALL NUMBERS INDICATE THE ORDER OF PROPOSING. 


Figure 4.21 
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where the De ie . 8 and oma are computed from the 25 
intensities of the k-th Sub-region scan segment according to 
(3.25), and from the paradigm intensity profiles illustrated in 
Figure 4.7. According to the discussion in Section III, the 
values in the feature point rasters are derived from the 
values given by (4.1) and (4. 2), and the present line verifier 
draws upon exactly these values. In this section we shall 
review the relevant theory in detail, describe the present line 
verifier, and suggest possible improvements and generalizations 
of it. 

The present line verifier, in computing a variant of 
Qus4) for a region, obtains the values of, e.g., ae : for 
CL(j) of (3.29) from the feature point rasters such as are 
partially depicted in Figure 4.10. In principle, if the region 
encloses n 25 point sub-region scan segments of the original 
scans, then the values ae Ke? etc. , Should be computed from 
exactly these scan segments. These values were computed 
for all possible sub-region scan segments, in the generalized 
manner given by (4.1) and (4. 2), by the feature point generation 


procedure. ftlowever, most of the values of the R's and R''s 
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were discarded during the compactification process. In fact it 
was only locally maximal values of E, given by: 


= iW = * ; ! 
Ey Ri ik Nn es He! (4. 3) 


where R, k and R' : are given by (4.1) and (4.2), that were 
’ J» 


retained. However, it turns out that we may obtain a 


reasonable estimate of the value of E, k for the intensities of 


a particular sub-region scan segment along a particular scan 
from one of these maxima nearby along the same scan. In 


particular, if E. 


ik is a desired value of E for a sub-region 


A 
scan segment along some scan, then let a k be the value of 


E which is locally maximal along the same scan and is computed 
from a sub-region scan segment closest to the one in question. 
We assume that there exists a maximum within something like 
15 units of the segment in question, else we assign ee the 
value 0. If Pig so defined actually exists, then the following 
relationship holds: 


A 2 
=~ = . 
- ced Be ax (4 4) 


E 
where ais some constant, and x is the distance between the 


df 
an ye WETS 


3 


sub-region scan segments from which » " 
computed. This relationship follows from the behavior of the 


E curve in the neighborhood of a maximum, which may be 
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observed in Figures 3.9 and 3.10. It will be seen later that 


the n values of E. 


ik are all that are necessary to compute the 


present variant of Q* used in the line verifier. 

This present variant of Q* omits the second and third 
terms in the exponent of, e.g., the value of CL as given by 
(3.29), and thus does not need the values of a Ke 2% and 
aye: The middle term may be ignored because it turns out to 
be always very small relative to the other two. To See this, 
recall that line verification in the present set of programs is 
applied only to lines, or possible lines, which escaped 
detection by the LINES program. They are thus lines which 
have a net relative amplitude whose magnitude is small relative 
to the average magnitude of relative amplitudes for lines in 
general. One may observe that the denominator of the middle 
term is dominated by its third term, so the term is approximately 
( zat? /n)/er, This is approximately the square of the net 
amplitude divided by its variance. Since the net amplitude 
in situations under consideration is small relative to the 
average magnitude of the net amplitudes, and the average net 


amplitude over lines in general is zero; then the net amplitude 
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is small relative to its standard deviation. The second term of 
(3. 29) is thus in this case very small, being approximately the 
square of the quotient of the net amplitude with the standard 
deviation. 

The omission of the third term of the exponent of (3. 29) 
was based on the empirical observation that for the majority 
of the scenes analyzed, this value seemed not to be related in 
a reliable way to the existence of lines. A series of experiments 
was begun to properly test whether any such relation existed. 
The results appeared promising but not so far conclusive. 
Consequently the third term was omitted pending further 
investigation. 

The resulting variant of Qx(J5) is quite similar to the one 
discussed at the end of the previous chapter. It is given by (3. 13), 
where the various values, e.g., of CL are given by (3. 29) but 
with only the first term in the exponent. This was the case in 
the situation described in the end of chapter three. Also, this 
time we expect a scan of 500 points to intersect with about five 
lines in the visual field, so the relation between P(CL), P(CE) 
and P(CH) remains approximately the same as in the situation 


at the end of chapter three. Finally, we may as before 
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assume that one of the terms in the numerator is considerably 
larger than the other. Thus by an argument similar to that 
used in deriving (3. 41) and (3. 43) we arrive at the function 

n : 
Q'**(T j) defined by: 


Qre*(Si) = 

n 

>) Max(SL, (j)-SH, (j), SE, (j)-SH(j)) + K, (4. 5) 
K=1 . = 


where, e.g., SL, (j) is like SL(j) given by (3. 37) with subscript 
k in place of 1. As before, this function is approximately 
monotone in Q*(J5) and hence thresholding it provides an 
approximately sub-optimal regional decision predicate. One 
may see from (3. 37), (3. 38), (3.39), (3.23), (3.24), (3.25) 
and a certain amount of algebra that: 

SL, (3) - SH, (j) = Rik - Rie! 
and 

SE, (j) - SH, (j) = Rik - Rik’ 
Using (4. 3) we may rewrite (4. 5) as: 


n 
Tskeok fe = 
are) = DIB + K oe 


with the values of ae iy obtainable via (4. 4) from values 


’ 


A 
z kK which are available in the feature point raster. In 
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summary we have: 

Qre(T2) = TB, y+ axk + K, (4.7) 

| .? 0 

recalling that if a Suitably close feature point for obtaining a 
value E, k did not exist, then the corresponding a4 was 
assigned the value zero. 

In practice it was found to give better results to 
deviate slightly from a strict thresholding of (4. 7) in the 
verification of lines. Prior to applying (4. 7) to a region it was 
observed which of the By, , were given nonzero values. In the 
case that the locations of the nonzero ee x 8 were sparse or 
concentrated in one place in the region, the thresholding 
procedure was omitted, and a negative answer was returned 
as to the existence of aline. This is approximately the same 
as observing the pattern of the locations of feature points within 
the region, and denying the existence of a line if they are all 
clumped together or are sparsely scattered within the region. 
These two situations are usually due to the existence of feature 
points from lines that cross the region at non-negligible angles. 
Another variation from (4. 7) which was found to be more 


convenient, and in Some cases more accurate, was to 
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A 
threshold the sums of the Ej , |S and the sums of the x's 
separately. Finally, instead of taking x, to be the deviation of 


A 
the E. , feature point from the center of the region, it was taken 


j,k 


A 
to be the deviation of the E. 


ik feature point from a best-fit line 


to the various E. k feature points. 

The line verifier (VERIFY Pl P2) takes as arguments 
the end-points Pl and P2 of the proposed line and outputs 
either NIL, if the line is thought not to exist by the above 
criterion; or the end-points of the best fit line to the feature 
points if the line is considered to exist. 

In observing the line verifier in action in Figure 4. 21, 
occasional failures are apparent. Although no lines are claimed 
to exist which do not exist in the figure, several actually 
existing in the figure were proposed but claimed, on the basis 
of the feature point raster information, not to exist by the 
verifier. An examination of the feature point raster indicates 
the presence of feature points in these regions. The problem 
was principally that the region intersected with too few scans, on 
the order of three to five; giving too few values for the threshold 
function (4. 7) to work with. Evidently a finer scan would 
eliminate this problem, as well as some other approaches to 
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its improvement which are about to be discussed. 


A major source of improvement for the line verifier is 


for it to obtain a new set of intensities within the region to which 


it is applied, at considerably higher density than that of the 
original scan. This would eliminate the difficulty described in 
the previous paragraph, would allow more accurate values of 
the R's, E's or whatever, to be obtained, and would eliminate 
the necessity of the approximation to the Et ,'S in (4. 4). 
Another improvement would result from the utilization of the 
third term of (3.29) and similar expressions. As was stated 
previously, investigations in this area are still under way. 

It seems plausible that the model in Section IV of chapter three 
may have to be modified in assumptions 4) and 5) to admit of a 
regular linear variation of "idealized" relative amplitude along 


a noise-free ridge-like or cliff-like noise-free sample. 
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CHAPTER 5 


RELATED WORK 


I OTHER APPROACHES TO LINE FINDING 


An early line-finding program was written by Roberts 
(Roberts). This program analyzed photographs, of good 
contrast and focus, of scenes whose content was principally 
defined by straight edge-lines. Feature points were obtained 
from a predicate on four adjacent intensities of a 2 by 2 square 
on the visual field. Feature points were organized into lines 
by a sort of line-following procedure. Higher level heuristics 
handled the resulting lines. 

A comparison may be made between the behavior of 
Roberts' four-point feature point predicate, and the present 
feature point gathering procedure. This analysis is a 
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generalization of a procedure due to Binford and Herskovits 
(Herskovits). 

An upper bound on the performance of the former may 
be inferred from theoretical considerations. Data on the actual 
performance is not available. The feature point predicate is 


applied to intensities x. » and x ; at four 


i,j? “ita, 3? “4, jet 
adjacent points in a square pattern on a 256 by 256 point grid. 
The thresholded value is: 

| ( a 2 + ( --Vs 2, 

iy . Yi, j itl, j+1 Yit1, j-Vi, +1)» 

where 

Yi,j “VS 7° 
Assume that the noise is gaussian, and that the noise level, as 
measured by the standard error about a mean of values of y 
taken at a single point under constant lighting conditions, is s 
for intensities in the vicinity of the average of the four y's in 
question. It is not difficult to show that z as a function of the 


random variables y; j has a 


> Vis, jp Vi, 41 299 Yivs, ji 
median of about 2s, and an upper interquartile point value of 
about 3s. In the context of the set of figures we are considering, 


one would expect about one feature point per 100 quadruples of 
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intensity. Thus for about 50% false positives, one would wish 
to set the threshold on z high enough that z falls above threshold 
by chance once in 100 times. For a normal distribution, this is 
about 2.5 standard deviations, which corresponds to 3 times the 
semi-interquartile range. Thus z should cut off at about 5s. 
Now it is not difficult to see what values of z would result for 
various lines and edges of amplitude a oriented in various ways 
with respect to the grid of squares. In Figure 5.1 we have 
computed such values for 12 cases. Inthe diagram, we are 
assuming that the dimensions of the blurring function are such 
as to give the line and edge profiles of the width shown. This 
width appears to be about an optimum compromise. If 
considerably lower then more edges would have a relatively 
high z-value, but more lines would have a zero z-value. As 

the width increases, both lines and edges tend toward having a 
zero z-value. From the diagram it may be estimated that the 
average value of z for edges is about . 8a, and for lines about . 4a. 
Thus the amplitudes of lines and edges respectively which are at 
threshold are about 6.3s and12.6s. This may be compared 
with the values 1. 2s and 2s obtained for Q* at the end of 


chapter three. The difference is due to two factors. First, 
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the Roberts predicate has a support of only four points, whereas 
the Q* tested had a support of 25 points. One would expect the 
signal to noise ratios to differ by a factor of ¥25/4 = 2.5. 

The actual difference is about 5, and the remaining factor of 
two is probably due to the fact that the Roberts predicate is 
isotropic, whereas Q* is directional. 

The Topologist, based on the work of Binford, Sussman 
(Sussman) and Herskovits (Herskovits), represents another 
approach to the analysis of simple scenes. In this case, the 
scene is divided into a set of small square regions which are 
individually tested for homogeneity. Adjacent homogeneous 
regions are clustered into maximal homogeneous regions, which 
are presumably the areas of the scene which lie between edge 
lines. This procedure has the properties that edge lines need 
not be straight, and that arbitrarily complex predicates for 
homogeneity may be used. 

Recently, a method of spatial frequency filtering and 
matching has been applied to the problem of location of feature 
points (Hueckel). This procedure is roughly a two dimensional 
version of the feature point analysis presented here. Fora 
circular neighborhood, an estimate is given of the directional 
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and amplitude parameters of a possible edge through the 
neighborhood. This procedure seems to be based on a rather 
simple model of the nature of intensity profiles at edges. Also, 
it seems to be relatively insensitive to the presence of lines. 

An approach bearing some mathematical similarities to 
the feature point detection procedure reported here, was 
employed by Saunders (Saunders) in an analysis of filmed 
images of scope tracings. In this case it was necessary to 
locate the intersections with parallel scan lines of the image of 
a narrow trace along the film. 

Worthy of mention here, though not specifically related 
to line finding, is the work in statistical decision theory of 
Chow (Chow). The theorem in the first chapter is essentially 
an adaptation and generalization of Chow's results in the theory 
of optimal error reject trade-off. 

II. RELATION OF THE PRESENT WORK TO THE 

DEVELOPMENT OF A VISUALLY ORIENTED REAL- 

TIME OBJECT MANIPULATOR 

The program here described is part of a larger effort to 
develop the visual-perception aspects of a real-time object 
manipulator. (MAC Progress Report 1968) For the present, 
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effort has centered on programs to obtain descriptions of scenes 
composed of cubes and other prismatic solids. By a description 
of an object is meant a list of the spatial co-ordinates of its 
vertices, together with a list of pairs of vertices defining edges 
of the object. By a scene description is meant a list of object 
descriptions, one for each object in the scene. 

There is an obvious disparity between the output of the 
present program and a corresponding scene description. The 
former is, or can easily be reduced to, a set of vertices and a 
set of vertex pairs representing edge lines. A description 
requires that these vertices and vertex pairs be grouped into 
sets corresponding to the various objects in the scene. One 
may regard the output of the current program as a set of lines 
dividing the field into regions. The problem then becomes one 
of grouping regions which correspond to faces of a single object, 
and matching regions which may be parts of a single face which 
has an occluding object in front of it which divides it into two 
regions. This problem has been investigated in detail, and a 
program called SEE (Guzman) performs the described grouping 


and matching to reduce the output of the present program into a 


175 


proper two dimensional scene description. 

Several approaches to the problem of determining the 
locations of objects in three dimensions have been investigated 
in greater or lesser degrees of detail. One suggestion involves 
the use of two light sources located close to the point of view of 
the optical input device. One would be a point source, whose 
intensity falls off as the square of the distance from the point of 
view. The other would be a source placed "at infinity'’ by means 
of lenses, whose intensity would fall off negligibly with intensity. 
If illumination is rapidly alternated between these two sources, 
and intensities obtained for a particular point under both 
conditions, the relation between the two intensities would yield 
the distance from the point of view to the point in the field. 
Another approach to determining the distance from the point of 
view to a point in the scene involves the use of optical focusing. 
(Horn 1968) By this procedure, a point in the scene is brought 
into perfect focus and the position of the lens of the optical input 
device is transmitted to the computer. After a suitable 
calibration procedure, the absolute distances of objects may thus 
be obtained automatically. A third approach (Horn) involves the 


use of intensity gradient to infer the shape of the contour of a 
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curved surface. By this technique, relative distance information 
may be obtained for points on curved surfaces. A final approach 
worked out by the present author, involving an analysis of the 
scene from two different points of view, relates directly to the 
current work. It is assumed that the present program has 
examined a Suitable scene from two different points in space, 
and that the resulting two sets of lines have been processed by 
SEE. Central to this approach is a subroutine called WHERE 
which is based on results in projective geometry, and is 
described in a recent Project MAC Artificial Intelligence Memo 
(Minsky). This program takes as input the two dimensional 
co-ordinates of a point from each scene. In many cases, WHERE 
can determine that the two points are not images of the same 
point in space. If the points are from the same point in space, 
WHERE outputs the three-dimensional co-ordinates of that point. 
One might apply this program in a straightforward manner to a 
point in one scene together successively with all points of the 
other. This would greatly reduce the possible ''matches' of 
features in the two scenes, but would, in general, not suffice to 
determine the three-dimensional locations of all the vertices. 
The problem of resolving the remaining ambiguities, as well as 
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developing a more efficient point-matching procedure seems to 
be quite complex. It seems likely that the organization that 

SEE imposes in the lines would provide a basis for using 
contextual clues to match the vertices. The problem is currently 
under investigation. 

A final problem worthy of mention is that of the 
determination of the stability of a configuration of prismatic 
solids, given a three-dimensional scene description of it. 
(Blum) The relevance of this problem derives from the fact 
that it is one of the goals of the object manipulator project to 
be able to construct structures from rectangular blocks. In 
order to direct the object manipulation device, a program 
must first determine the order in which the blocks are to be 
set down. It may be possible to construct a sub-configuration 
of the desired configuration which has the property that adding 
any additional block results in an unstable configuration. To 
forestall such a possibility, it is necessary to be able to 
detect instability in searching for an appropriate order in which 


to build the structure. 
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